HOME

TheInfoList



OR:

''DevOps Research and Assessment'' (abbreviated to DORA) is a team that is part of
Google Cloud Google Cloud Platform (GCP) is a suite of cloud computing services offered by Google that provides a series of modular cloud services including computing, data storage, data analytics, and machine learning, alongside a set of management tools ...
that engages in
opinion polling An opinion poll, often simply referred to as a survey or a poll, is a human research survey of public opinion from a particular sample. Opinion polls are usually designed to represent the opinions of a population by conducting a series of que ...
of
software engineers Software engineering is a branch of both computer science and engineering focused on designing, developing, testing, and maintaining software applications. It involves applying engineering principles and computer programming expertise to develop ...
to conduct research for the
DevOps DevOps is the integration and automation of the software development and information technology operations. DevOps encompasses necessary tasks of software development and can lead to shortening development time and improving the development life ...
movement. The DORA team was founded by Nicole Forsgren, Jez Humble and Gene Kim. and conducted research for the
DevOps DevOps is the integration and automation of the software development and information technology operations. DevOps encompasses necessary tasks of software development and can lead to shortening development time and improving the development life ...
company
Puppet A puppet is an object, often resembling a human, animal or Legendary creature, mythical figure, that is animated or manipulated by a person called a puppeteer. Puppetry is an ancient form of theatre which dates back to the 5th century BC in anci ...
and later became an independent team (with Puppet continuing to produce reports by a new team). Whilst the founding members have departed, the DORA team continue to publish research in the form of annual ''State of DevOps Reports''.


State of DevOps Reports

The DORA team began publishing State of DevOps Reports in 2013. The latest DORA State of DevOps Report, published in 2024, surveyed "more than 39,000 professionals across many industries globally." It found AI is having a broad impact on software development, with largely positive results. "Flow," "productivity," and "job satisfaction" are cited among 8 benefits of AI adoption for software developers. The detrimental effects of AI were "reductions to software delivery performance" and "a reported decrease in time spent doing valuable work."


DORA Four Key Metrics

For the purposes of their research, Four Key Metrics, sometimes referred to as DORA Metrics, are used to assess the performance of teams. The four metrics are as follows: * Change Lead Time - Time to implement, test, and deliver code for a feature (measured from first commit to deployment) * Deployment Frequency - Number of deployments in a given duration of time * Change Failure Rate - Percentage of failed changes over all changes (regardless of success) * Mean Time to Recovery (''MTTR'') - Time it takes to restore service after production failure Using these performance measures, the team are able to assess how practices (like
outsourcing Outsourcing is a business practice in which companies use external providers to carry out business processes that would otherwise be handled internally. Outsourcing sometimes involves transferring employees and assets from one firm to another ...
) and risk factors impact performance metrics for an engineering team. These metrics can be crudely measured using
psychometrics Psychometrics is a field of study within psychology concerned with the theory and technique of measurement. Psychometrics generally covers specialized fields within psychology and education devoted to testing, measurement, assessment, and rela ...
or using commercial services.


Limitations

These metrics have been used by organisations to evaluate team-by-team performance, a use-case which the DORA team issued a warning against in October 2023. Some professionals have argued that using the DORA Four Key Metrics as a target within engineering teams encourages focus on wrong incentives. For example; James Walker, CEO at Curiosity Software, has argued the "metrics aren’t a definitive route to DevOps success" and challenges in using them for team comparisons. In addition, Steve Fenton, Director of Developer Relations at Octopus Deploy, suggests an additional metric: Reliability. Reliability refers to teams meeting or exceeding their reliability targets. The State of DevOps Report research finds that operational performance drives benefits across many outcomes. Measuring reliability ensures that end-user experience gets included. The example includes the quality of the internal documentation, which is a key to high performance against the reliability metric. Teams with high-quality documentation were more than twice as likely to meet or exceed their targets. Documentation also improved performance against the other DORA metrics. Research conducted by the computer scientist Junade Ali and the British polling firm Survation found that both software engineers (when building software systems) and public perception (when using software systems) found other factors mattered significantly more than the outcome measures which were treated as the "Four Key Metrics" (which ultimately measure the speed of resolving issues and the speed of fixing bugs, and are used to create the findings in the book), and risk and reward appetite varies from sector-to-sector. Ali has also criticised the research on the basis that reputable opinion polling firms who comply with the rules of organisations like the
British Polling Council The British Polling Council (BPC) is an association of market research companies whose opinion polls are regularly published or broadcast in media in the United Kingdom. The current President is Jane Green. The BPC was established in 2004, ...
should publish their full results and raw data tables, which the DORA team did not do - and additionally that the sponsors of the polling (Google Cloud and previously Puppet) create products which have a vested interest in having software engineers deliver faster (despite research indicating high levels of burnout amongst software engineers), which the results of the research ultimately supported. Despite the authors arguing that speed of delivery and software quality go hand-in-hand, Ali has offered several counter-examples; including the comparatively high quality of aviation software despite infrequent changes, contrasted with
rapid application development Rapid application development (RAD), also called rapid application building (RAB), is both a general term for adaptive software development approaches, and the name for James Martin's method of rapid development. In general, RAD approaches to ...
being pioneered in the software that resulted in the British Post Office scandal and
agile software development Agile software development is an umbrella term for approaches to software development, developing software that reflect the values and principles agreed upon by ''The Agile Alliance'', a group of 17 software practitioners, in 2001. As documented ...
being used in the software responsible for the 2009–2011 Toyota vehicle recalls. The software developer Bryan Finster has also discussed how, as
correlation does not imply causation The phrase "correlation does not imply causation" refers to the inability to legitimately deduce a cause-and-effect relationship between two events or variables solely on the basis of an observed association or correlation between them. The id ...
, organisations who are considered "high performing" in the research are not high performing because they focussed on the DORA metrics, but instead focussed on delivering value to users and arguing the metrics should be used as "trailing indicators for poor health, not indicators everything is going well".


Accelerate (book)

''Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations'' is a
software engineering Software engineering is a branch of both computer science and engineering focused on designing, developing, testing, and maintaining Application software, software applications. It involves applying engineering design process, engineering principl ...
book co-authored by Nicole Forsgren, Jez Humble and Gene Kim from their time in the DORA team. The book explores how software development teams using Lean Software and
DevOps DevOps is the integration and automation of the software development and information technology operations. DevOps encompasses necessary tasks of software development and can lead to shortening development time and improving the development life ...
can measure their performance and the performance of software engineering teams impacts the overall performance of an organization. The book discusses their research conducted as part of the DORA team for the annual ''State of DevOps Reports''. In total, the authors considered 23,000 data points from a variety of companies of various different sizes (from start-up to enterprises), for-profit and not-for-profit and both those with legacy systems and those with modern systems.


24 Key Capabilities

The authors outline 24 practices to improve software delivery which they refer to as "key capabilities" and group them into five categories.


Continuous Delivery

* Use version control for all production artifacts * Automate your deployment process * Implement Continuous Integration * Use trunk-based development methods * Implement test automation * Support test data management * Shift Left on Security * Implement Continuous Delivery (CD)


Architecture

* Use a Loosely Coupled Architecture * Architect for Empowered Teams


Product and Process

* Gather and Implement Customer Feedback * Make the Flow of Work Visible through the Value Stream * Work in Small Batches * Foster and Enable Team Experimentation


Lean Management and Monitoring

* Have a Lightweight Change Approval Processes * Monitor across Application and Infrastructure to Inform Business Decisions * Check System Health Proactively * Improve Processes and Manage Work with Work-In-Process (WIP) Limits * Visualize Work to Monitor Quality and Communicate throughout the Team


Cultural

* Support a Generative Culture * Encourage and Support Learning * Support and Facilitate Collaboration among Teams * Provide Resources and Tools that Make Work Meaningful * Support or Embody Transformational Leadership


References

{{Reflist


External links


CLOUD AND DEVOPS CONSULTING

QCon Plus (May 17-28): Stay Ahead of Emerging Software Trends Q&A on the Book Accelerate: Building and Scaling High Performance Technology Organizations
Computer programming books Computer books