Agile methodologies had gone from obscurity to buzzwords, and in recent years became mainstream. That does not mean teams and companies are doing it well, however, despite enormous amount of time and money spent on training and software such as Application Lifecycle Management and Continuous Integration tools. The old motto of “you can’t manage what you can’t measure” is as true in traditional waterfall teams as modern agile squads, although what you should be measuring are vastly different.

The basic premise of agile is to increase communication to shorten feedback cycles and minimize waste. Teams eschew lengthy written documentations and instead have face-to-face conversations about just enough requirements to produce working software in weeks. They stand around story index cards and sticky notes covering acres of wall every day to talk about their work in progress. It is easy for senior leaders who are not involved with everyday delivery to think this is all agile is about. Mature agile teams care about deployment frequency, lead time for changes, change failure rate, and mean-time-to-recover (MTTR) instead. These metrics, together, measure how capable teams are to deliver software to market in a high-quality fashion very quickly and sustainably.

Deployment frequency measures how often teams deploy fully-tested, working software that is valuable to end users. Deploying at a high frequency requires automating the workflow of compiling, packaging, testing, integrating and deploying software using a deployment pipeline. Techniques such as infrastructure-as-code ensure production and earlier environments are identical, thereby turning production deployments into a well-rehearsed, repeatable process. If done well, a deployment pipeline changes production deployments from a dreaded, weekend-long IT exercise involving dozens of people to a business-led decision that can be done on-demand.

Once a deployment pipeline is in place, high-performing teams refine and optimize it to bring down the time required for a code commit to run in production, known as lead time to change. After all, a fully-automated deployment pipeline that takes days to run still means deployments can be done only once every few days. Other agile techniques such as Minimum Viable Product and vertical thin-slicing also help cut down the time from idea to cash.

A little understood fact of software development is testing as a separate phase after development does not improve software quality. The fix to a software defect so late in the cycle may require redesigns so drastic that it demands more time than can be afforded. Low-performing teams put in shortcuts to fix just the most egregious problems. Over time, such corner cuttings accumulate, obscure the true intent of the software and weaken its structural integrity. Instead, high-performing teams build quality and security into their software using techniques such as Test-Driven Development (TDD) and secure coding practices. At its core, TDD is both a design technique whereby developers specify how the software shall behave, as well as a testing practice to ensure teams are informed of breaking changes in the codebase as soon as a defect is introduced. Secure coding practices such as input validation and output encoding help prevent expensive and embarrassing security breaches. These techniques empower teams to make code changes fearlessly while at the same time reduce change failure rate.

Alas, the truth is even high-performing teams suffer production issues and bugs that are found only in production by their users. What sets high-performing teams apart is their engineering discipline and well-rehearsed release process that enable them to reproduce defects and deploy fixes rapidly. Dysfunctional teams tend to patch production systems directly in emergency situations because their official release procedure is manual, tedious and unreliable. Lowering your team’s MTTR is another reason to adopt a DevOps culture and implement techniques like deployment pipelines.

A positive side-effect of an automated pipeline linking requirements to code commits, through signed artefact and authenticated deployments is its auditability. Many organizations and industries require traceability of production deployments. A strong DevOps culture and automated delivery pipeline provide Compliance-as-Code.

Most teams start their agile journey by using the ceremonies and rituals from Scrum. A common metric enterprises measure is percentage of their IT staff holding Scrum certifications or number of projects ‘doing agile.’ More and more large enterprises begin to realize that is not enough. To compete and survive in today’s digital economy, they also have to transform themselves into technical companies and bring back outsourced technical capabilities in-house. That calls for different measures of success. Measuring agile teams the right way will foster a kaizen culture in a virtuous cycle that benefits not just the IT teams but the organization as a whole.