score

4 Levels Of Agile Metrics / 10 Steps that Amazon Takes to Measure Impact [#Agile #AgileMetrics #MeasureImpact]

4 Levels Of Agile Metrics
1. “A good idea”
2. “An output”
3. “An outcome”
4. “The impact”

10 Steps that Amazon Takes to Measure Impact
1. An Obsession with Customer Value, not Shareholder Value
2. Shared Responsibility for Customer Focus
3. Customer-Focused Metrics
4. The Metrics are Built on a Narrative
5. Activities Report to the Organization, Not the Unit
6. The Organization Budgets Activities, Not Units
7. Work is Done in Small Teams Working in Short Cycles
8. Budgeting is a Subset of Planning, not Vice Versa
9. Controlling Spending vs. Managing Impacts
10. Relation to Human Resources

1. “A good idea”:
An activity that is undertaken because enough influential people believe that it is likely to have some benefits.

These are often, as Scrum.org points out, “really just conjectures about what customers might like (sometimes referred to as HiPPOs, or Highly Paid Person’s Opinions) … The specification of the solution is vague and imprecise…”

At worst, they are the hobby horse of some organizational faction promoting their own particular interest.

2. “An output”:
Something internal, measurable but not necessarily related to any external customer.

This is better than a mere conjecture that it is a good idea, but still not getting the organization very far in terms of understanding the activity’s value.

3. “An outcome”:
Something external such as customer satisfaction in relation to value delivered.

It is often subjective and vague and fuzzy.

The Net Promoter Score, which fits into this category, has been shown to be positively correlated with actual impact and is certainly better than not having any measure, but its meaning can be ambiguous and difficult to read.

4. “The impact”:
Changes in customer behavior that the product or service is intended to elicit.

This goes beyond merely whether the customer buys the product or service and may include measures of actions—or non-actions—that you would expect if the customer is truly delighted, such as timely availability of the item, speed of delivery, percentage of unexpected “hiccoughs” in delivery, absence of returns and complaints, re-purchases of the product and related products, responses to surveys recommendation of the products to other customers, and so on.

10 Steps that Amazon Takes to Measure Impact

1. An Obsession with Customer Value, not Shareholder Value
   Everything begins with an obsession with delivering value to customers. Shareholder value is the result, not the goal.

Executive compensation at Amazon is aligned with these principles. Executives have relatively modest salaries plus stock. Their compensation is not differentiated in terms of the performance of their own units.

2. Shared Responsibility for Customer Focus
Customer obsession is not just for top management, sales, and marketing.

Everyone is expected to be obsessed with knowing about and enhancing the impact of what they do for the customer.

3. Customer-Focused Metrics
In most big organizations, customer data—where it exists at all—is spotty and soft, while financial data is hard and ubiquitous.

The hard financial numbers consistently trump any soft customer numbers.

Not so at Amazon, where real-time customer metrics are built into every aspect of the work. As a result, “customer value” wins battles that it would not win in firms run with a financially oriented mindset.

4. The Metrics are Built on a Narrative
No significant new activity can be undertaken at Amazon unless and until there is an exhaustive management review of a 6-page document explaining the activity as a narrative.

The 6-page narrative is supported by another document known as the PR/FAQ, which contains an imagined future press release describing the benefits customers are getting and answers to “frequently asked questions” about how the activity was developed.

The metrics by which customer benefits of the activity will be measured in real-time are based on this narrative.

5. Activities Report to the Organization, Not the Unit
In most big organizations, in the absence of customer obsession, one can easily imagine that a planning process as at Amazon would quickly degenerate into a battle between units fighting for resources for “their” activities and “their” units.

Unlike most big organizations, executives at Amazon are not accorded prestige or salary according to the size of their staffs or their budget.

6. The Organization Budgets Activities, Not Units
What often makes traditional budgeting so frustrating is that everyone knows that substantive issues about purpose and priorities are not being addressed in the budget discussion: the budget battles are a proxy for unresolved power struggles between competing units and the divergent viewpoints of senior managers.

Traditional budgeting often reflects, reinforces and aggravates the siloed organizational structure. Siloes often get in the way of delivering value to customers.

Rather than having truly cross-functional teams, with end-to-end responsibilities to deliver value to customers, supposedly “Agile” teams may work within their own silo and produce outputs, for which they may get credit in the budgeting process, but which lead to no immediate customer benefit.

7. Work is Done in Small Teams Working in Short Cycles
As in all Agile management, work is done at Amazon to the extent possible in small, integrated autonomous multidisciplinary teams, known at Amazon as “two-pizza teams,” i.e. a team that you could in principle feed with two pizzas.

At Amazon, two-pizza teams work like semi-independent entrepreneurial hothouses. Insulated from the greater organization’s bureaucracy, the Two-Pizza Teams encourage ambitious leaders, provide opportunity, and instill a sense of ownership.”

8. Budgeting is a Subset of Planning, not Vice Versa
Budgeting at Amazon is a subset of an exhaustive planning process known as OP1 in which every activity is reviewed in terms of its contribution to value for customers.

At Amazon, the reviews principally revolve around real-time customer-related metrics of individual activities, not about which unit gets how much money.

That review is possible because each activity is evaluated at the very outset from the perspective of its eventual customer impact.

9. Controlling Spending vs. Managing Impacts
Traditional budgeting is predicated on careful monitoring of spending, often in the absence of reliable information about what is happening on the more important issue of customer outcomes.

Bad news is often kept from top management until it is too late to do anything about it.

At Amazon, the availability of real-time customer-related metrics at all levels ensures radical transparency.

There is nowhere to hide. Everyone knows everything.

10. Relation to Human Resources
The problems of traditional budgeting are often aggravated by the HR system of the organization, which may reflect and reinforce the siloed and hierarchical structure and budget, in which customer outcomes and impacts are not fully in the picture.

Thus, individuals within each organizational silo tend to be evaluated and rewarded in terms of producing the budgeted level of outputs within the given budget envelope for their unit.

The compensation structure at Amazon is focused on incenting long-term enterprise value creation.

The focus is on achievement (big and small) with responsibility for getting to yes which is everyone’s job.

Adapted from:
Why Agile Often Fails: No Agreed Metrics:
Forbes