Key Agile Metrics to Track [#AgileMetrics]

Key Agile Metrics to Track [#AgileMetrics]

1. Blocked time
2. Code coverage
3. Control chart
4. Cumulative flow diagram
5. Release Cycle time
6. Epic and Release burndown
7. Failed deployments
8. Lead time
9. Net promoter score (NPS)
10. Defect escape rate
11. Sprint burndown
12. Throughput
13. Value delivered
14. Velocity
15. Work item age

1. Blocked time
Blocked time is defined as the amount of time a particular user story or a task – is blocked.

Resolving blockers is critical to facilitating the flow of work in an agile environment. This metric can help measure the amount of time they take to resolve. Blockers should be resolved expediently.

Increases in blocked time could mean that a user story was not decomposed properly, or that there is a dependency on an outside resource that was unplanned. Blocked time can be reduced with more careful user story decomposition, prioritization and sprint planning.

2. Code coverage
Code coverage is a measure of how much of the code is actually being executed during testing.

This is typically instrumented and calculated as part of an automated testing strategy.

The metric should provide the overall percentage of code executed during each testing phase (unit, system, and so on) as well as a total of all phases.

Code coverage should not be misused as a marker for how well a product has been tested. Rather, the goal of this metric is to facilitate test automation and monitor continuous delivery.

3. Control chart
Sometimes referred to as a process-behavior or Shewhart chart, a control chart monitors the performance of a process to determine whether it is under control or out of control – depending on the upper, lower, and mean control limits that have been set.

These limits are calculated by estimating the standard deviation of the sample data, multiplying that deviation by three, and then adding that to the average to create the upper limit, and subtracting it from the average to create the lower limit.

The Y-axis of the chart is the number of occurrences or issues in a particular sample while the X-axis enumerates each sample.

Popular with six sigma disciples, control charts can measure the failure or success of quality control or other manufacturing processes.

While not popularized in the agile world, control charts could be used to measure defects found per iteration or release to identify QA issues, or to gauge cycle times over a series of releases to ensure that they fall within acceptable levels.

4. Cumulative flow diagram
A cumulative flow diagram illustrates how much work, segmented by type, is being allocated to a team over time.

Its purpose is to monitor how well work is flowing throughout the system.

In this diagram, work is divided into different types, for example: to do, in progress, and done. It could also be divided into requirements, development, testing, and so on.

However it is segmented, the cumulative flow diagram shows a line for each work type, with the number of work items on the Y axis and the X axis being a function of time.

Good flow is illustrated by all of these lines running in parallel. If one of the lines experiences a sharp upturn, or crosses over another, this could indicate a bottleneck.

Achieving good flow is the central concept behind Kanban.

The cumulative flow diagram helps identify bottlenecks to facilitate continuous flow and ensure that WIP does not get out of control at any one point in the system.

5. Release Cycle time
Release cycle time can be defined as how long it takes to produce a software release, from concept to completion.

Along with lead time and velocity, release cycle time is a very good high level indicator of agile health and agile transformation success.

As an organization progresses in its agile journey, cycle times should gradually decrease, typically to 6 months or much less. Increases in cycle time, especially when observed consistently over one or two releases, should be a cause for concern and review.

6. Epic and release burndown
Epic and release burndown charts are similar to the ever-popular sprint burndown.

A burndown chart illustrates how much work remains for a given time period or for a certain epic.

In agile development, an epic is a larger user story composed of smaller user stories or chunks of work.

As work is completed, the number of user stories in the epic is gradually reduced until it reaches zero. This can be useful in cases where milestones must be reached to meet contractual requirements and bill the client.

Similarly, a release burndown can track progress of work committed for a specific release. This can be used to help ensure on-time delivery or identify a need to change a deadline early.

7. Failed deployments
A failed deployment is one that results in any of the following:

Service affecting outage
Fails to meet customer expectations, often resulting in rejection of the release.
Seriously affects usability, operation, or user experience of the product.
Results in a rollback to the previous release.

Obviously the failed deployment rate, displayed as a percentage of total deployments, should be kept to a minimum. Any spike in this metric should be cause for concern. Change rates and defect occurrences should be reviewed to isolate root causes.

8. Lead time
Lead time measures the time it takes to complete a task, from the moment it is created to the point where it is finished.

In short, it identifies how long it takes to get things done.

Popular with Kanban practitioners, this metric can help identify efficiencies to move tasks faster though the system. It can also be used as a high level metric for determining how well continuous delivery is working.

Lead time, along with cycle time and velocity, can be used together to provide a holistic view of delivery performance.

9. Net promoter score (NPS)
A net promoter score is intended to help rate customer satisfaction.

It is usually calculated based on data obtained via a survey. The goal is to find out how many customers would recommend your product. The percentage of respondents that vote “no” are subtracted from the “yes” voters to create the score.

In addition to gauging customer satisfaction, the net promoter score can help identify customers more willing to collaborate on innovative products or technologies for future releases. Such customers can become a competitive advantage as their feedback and support can help organizations get new products to market before the competition does.

10. Defect escape rate
Defects can be monitored based on where and when they occur, their frequency, and severity.

One of the most popular is the defect escape rate, which is the ratio of defects found by the customer to the total number of defects discovered in a release.

Although high numbers of defects should be concerning no matter how they are found, it is always best to catch them before the customer does.

11. Sprint burndown
Sprint burndown charts provide a daily measure of the work that is completed, and the work that remains to be done in a given sprint.

It compares the amount of work completed to the original estimates. Due to the empirical nature of agile development, the value of the burndown chart is quite limited.

Despite its popularity, many agile coaches are moving away from using it as much as before. It can serve as a good guide or status point for where development teams stand against their commitments, but it should be used in tandem with other metrics to get the whole picture of what is going on.

12. Throughput
The quantity of product (number of work items) delivered to the customer per a particular unit of time is referred to as throughput.

This could be measured monthly, quarterly, per release, iteration, and so on. The value in this metric is that it can be used to determine how much software can be delivered for a specific time frame. It can also be used to track the consistency of delivery from a team and organizational perspective.

Empirical analysis of historical data can be used to forecast delivery performance. The more historical data available, the more accurate the projections are likely to be.

Most crucially, this metric can also be used to forecast revenue, given that the value of delivered feature functionality is well understood in financial terms.

In order for this metric to work, the definition of “done” must be well-defined. Only software delivered to the customer meets this requirement.

13. Value delivered
Value consists of extrinsic quality or the perception of the product from the end user.

How does the product impact the customer’s business? Good agile metrics are based on outcomes, and in the business world, that typically translates into dollars and cents. Just as we assign story points to each user story as a way to estimate the work it takes, we can also add value points as a relative measure to indicate what the end user is getting when the work is done.

One way to do this is with a burn-up chart that illustrates the number of value points accumulated as each story is completed. Value points can be assigned to each story or feature based on customer perception as the acceptance criteria is being created. The expected revenue (or money saved) for the customer on the project can be divided by the total number of value points in the release.

For example, if there are 200 value points in a project and the customer is expected to gain 1 million dollars in revenue, then each value point is worth $5,000. The sum total of each story and their accumulated value can be illustrated in the burn up graph. Although the actual impact of the product may not be apparent until it is released, this method can provide compelling financial intelligence for management and customers alike.

14. Velocity
Velocity is probably the first metric most of us hear about after being introduced to agile development. Although arguably the most popular agile metric, it is also the most misused. Sprint teams are notorious for gaming velocity because it is so heavily relied upon for reporting their performance.

Velocity is defined as the quantity of software produced in each iteration, or sprint. This quantity is usually expressed as story points and the software produced must be a functional production ready slice of code.

Teams often game velocity by manipulating the size and estimation of user stories, or by decomposing work horizontally, instead of vertically, by creating stories for database changes, front end work, middleware, and more. to eliminate dependencies on others and get credit for completing work. The problem with this approach is that those kinds of user stories are really tasks, and although the teams get credit, business value for the customer has not been delivered.

Gaming velocity can be prevented by using a host of other metrics as a check and balance against each other. All too often organizations rely solely on velocity or a very small set of metrics instead of a larger suite of measurements to form a PPM, program and project management solution.

15. Work item age
A work item could be defined as a package of work, usable feature, or as it would be in most agile contexts, a user story. The clock starts ticking on a work item’s age as soon as it is first conceived. Tracking the age of work items, whether they are in progress or sitting in the backlog, can help identify issues with requirements.

If a work item seems to be getting older than its kin because it is being pushed off from one sprint to the next, there could be an issue with decomposition. Maybe it needs to be redefined or better understood? Work items that sit in the backlog for extended periods may need to be culled or redefined.

Continuous backlog grooming is critical to sprint planning and prioritization. A growing number of aging requirements in the backlog could mean trouble with how the requirements are being developed and decomposed. Poor requirements management is one of the primary causes for failure in agile transformations.

Badly written requirements can make prioritization and estimation extremely difficult, resulting in out of control technical debt, low feature utilization, and financial loss. Developing well understood, prioritized, high-value requirements is very much an art form and poorly understood by even the best of agilists. Indeed, it is arguably one of the biggest blockers to agile transformation success.

17 Important Agile Metrics Your Team Should Care About
G2 Crowd

Like this? Leave your thoughts below...

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Posted in:


Don`t copy text!