The metric that drove the wrong behaviour

In December 2024 a US Senate committee published a 159-page report saying Amazon had commissioned its own research into warehouse injuries, found a link between speed and harm, and set the findings aside. Amazon says the report is wrong on the facts. Strip out the politics and what remains is the oldest problem in improvement work: an organisation that measured one thing extremely well and got exactly what it measured.

Every organisation I have worked in has at least one measure that everybody privately knows is producing bad behaviour, and that nobody will touch because it is in the contract, or in the board pack, or in somebody’s bonus.

You can usually find it in under an hour. Ask any experienced operator: what do you do that makes the number look good and the work worse? They will tell you immediately, with examples, and they will be slightly surprised that anybody asked.

The reason this survives is not that people are dishonest. It is that measures are easy to install and almost impossible to remove, and that the damage they do is diffuse, delayed and lands on somebody other than the person who owns the number.

The case

On 16 December 2024 the majority staff of the United States Senate Committee on Health, Education, Labor and Pensions published a 159-page report on safety in Amazon’s warehouses. Committee staff interviewed 135 workers and reviewed 1,400 documents provided by workers; Amazon contributed 285 documents, and the report characterises its cooperation as limited.

Two pieces of Amazon’s own internal research sit at the centre of it.

Project Soteria, begun in 2020, was described by its facilitators as the first comprehensive analysis of the causes of warehouse injury at Amazon. It found a connection between worker speed and injuries — echoing a finding by Washington state regulators — and recommended pausing speed-related discipline and building more downtime into schedules. About two years in, Amazon’s senior safety executive commissioned another team to review it. That review questioned Soteria’s methodology and recommended more frequent safety meetings. Amazon says Soteria’s analysis was preliminary, outdated and inaccurate.

Project Elderwand, begun around 2021, examined how repeated movements over ten- and twelve-hour shifts affect the risk of soft-tissue musculoskeletal injury. It found that for workers picking items from robotic storage units, the likelihood of back injury rose with the number of items picked per shift. It proposed more work breaks. A small pilot was run, the report says, to avoid disrupting productivity. Amazon rejected the findings; a spokesperson called the proposed solutions ineffective and pointed to other ergonomic improvements the company had pursued.

The report’s framing is blunt: that Amazon accepted “injuries to its workers as the cost of doing business.”

Amazon’s rejection is equally blunt and belongs here. Spokesperson Kelly Nantel said the report “is wrong on the facts and weaves together out-of-date documents and unverifiable anecdotes”, said the company’s expectations of staff are safe and reasonable, and pointed to recent declines in injury rates. The company published a blog post rejecting the report. A Washington state judge dismissed high-profile workplace safety citations against Amazon, a decision the state labour regulator is appealing.

I am not in a position to adjudicate, and I am not going to pretend otherwise. But notice that the improvement lesson does not depend on who is right about the injury statistics.

Two internal studies asked whether a productivity measure was causing harm. Both concluded that it contributed. Both proposed changes that would have reduced throughput. Neither change was adopted at scale. A third study was commissioned, found methodological problems with the first, and recommended something that cost nothing — more safety meetings.

That sequence is not unique to Amazon. It is what happens in most organisations when evidence arrives that an installed measure is doing damage, and I have watched it play out over smaller stakes many times.

The pattern, generalised

A measure gets installed because it is countable, comparable and connected to something the organisation genuinely needs. Nothing wrong so far. Throughput matters. Average handling time matters. Utilisation matters.

Then three things happen.

It becomes the definition rather than the proxy. “Items picked per hour” starts life as a proxy for productive work and quietly becomes what productive work means. At this point you can no longer argue against the measure without appearing to argue against productivity itself, which is a rhetorical position almost nobody can hold in a management meeting.

It acquires consequences. Appraisal, ranking, discipline, contract penalties. Once a number has a consequence attached, the organisation has created a strong, distributed, continuous incentive to improve the number by any available route — including routes that damage the thing the number was standing in for.

It becomes commercially load-bearing. The number appears in a client SLA, an investor deck, a regulatory return. Now changing it is not an operational decision but a commercial one, requiring people who have never seen the work to agree. This is the stage at which most bad measures become permanent.

By the time somebody produces evidence of harm, the measure is three layers deep in the organisation’s commitments, and the evidence is competing against that.

Why counter-evidence loses

This is the part worth sitting with, because it is where improvement professionals get beaten.

The cost of the measure is diffuse: spread across many people, showing up months later, in a different function’s numbers — safety, attrition, quality, complaints. The benefit is concentrated: visible this quarter, in one owner’s number.

The evidence of harm is statistical and therefore contestable. There is always a methodological objection available, and it is often a fair one, because research inside a working organisation is genuinely messy. The Soteria review is a textbook instance: a second study found methodological weaknesses in the first. That may be entirely correct. It also had the effect of converting a finding into a dispute.

And the cheap intervention is always available. More meetings, more training, more communication, a poster campaign. These cost nothing, demonstrate responsiveness, and do not touch the measure. When a leadership team must choose between a change that reduces output and a change that does not, the one that does not will win unless somebody with authority forces the comparison.

So the practical question for anyone doing improvement work is not “how do I produce better evidence?” It is “how do I get the comparison made honestly, by somebody who owns both sides of it?”

Four defences that work

Pair every measure. No throughput measure without a quality or harm measure beside it, reported together, to the same person, at the same cadence. Not in a different pack. Not owned by a different function. Side by side, so that a movement in one is visible against the other. This is the single highest-value change available to most organisations and it costs nothing but an argument about report design.

Service desks: handling time next to repeat-contact rate. Manufacturing: output next to rework and near-miss. Sales: closed deals next to first-year churn. Collections: recovery rate next to complaint rate.

Put a tolerance on the paired measure, with teeth. A paired measure with no threshold is decoration. Say explicitly: if the harm measure crosses this line, throughput expectations are suspended until it comes back. Agree it before it happens, because afterwards it will be negotiated against a bad quarter.

Make somebody senior own the trade-off, not one side of it. Where operations owns throughput and safety or quality owns harm, the organisation has outsourced its most important judgement to a standing conflict between two directors, one of whom is more powerful. Somebody above both must own the ratio and be accountable for it by name.

Keep a register of known measure distortions. A one-page living document: this measure, this is the behaviour it drives that we do not want, this is what we do about it. Review it twice a year. The value is not in the entries; it is that the register legitimises saying out loud that a measure has side effects, which in many organisations is currently a career-limiting observation.

Designing a measure that ages well

Most of the measures causing trouble today were designed quickly, by someone solving a real problem, with no expectation that the measure would still be in force a decade later. A little care at the design stage buys a great deal.

Five tests I apply to any proposed measure before it goes into a scorecard or a contract:

The gaming test. Before it is agreed, sit down with three people who will be measured by it and ask them how they would make it look good dishonestly. If they cannot think of anything in five minutes, either the measure is unusually robust or you have the wrong three people. Write their answers into the measure’s definition as explicit exclusions.

The attribution test. Can the people being measured actually influence it, within the time period being measured? A measure that moves mostly because of things outside the team’s control teaches people that effort is unrelated to outcome, which is corrosive in a way that is hard to undo.

The pairing test. What is the harm this measure could cause if someone optimised it ruthlessly? That answer is the paired measure. If you cannot name one, you have not thought about the measure hard enough.

The expiry test. Why is the target at this level, who set it, and when should it next be re-derived? Write the answer down beside the measure. Measures without a stated rationale become permanent by default, because nobody can argue against a number whose origin nobody remembers.

The decision test. What decision will change depending on this number? If the honest answer is “none, we just report it”, you have added cost and risk for no benefit. Retire it. A scorecard of twelve measures where four drive decisions is worse than a scorecard of four.

That last one deserves emphasis. Most organisations have far too many measures, and the excess is not harmless — it dilutes attention, multiplies the surface area for gaming, and makes it much harder to see the one or two numbers that are genuinely moving. Removing measures is improvement work. It rarely gets counted as such.

When you cannot change the measure

Often you cannot. It is in a client contract, or a regulatory return, or a group-wide scorecard set three time zones away. Three things still available:

Change what is excluded from it rather than the measure itself. Verified identity checks excluded from handling time. Genuine safety stops excluded from throughput. Training time excluded from utilisation. This is far easier to get agreed than a target change and it removes most of the perverse pressure.

Change the unit of accountability. Move the measure from the individual to the team. Almost all the worst gaming behaviour comes from individual-level measurement of work that is actually collective, and moving the level of accountability frequently removes the distortion without touching the number.

Change the review, not the target. If the number is fixed, change what happens in the conversation about it — who attends, what else is on the table, what evidence is required to explain a movement. A target reviewed alongside its harm measure behaves very differently from the same target reviewed alone.

A note on Indian operations

Two things I see repeatedly here.

The first is inherited measurement. A large share of Indian delivery operations run against measures written by a client, in another country, for a context that no longer exists — targets set at transition and never revisited, sometimes for a decade. Nobody on either side can explain why the number is what it is. These are frequently easier to change than people assume, because the client’s own operations team often dislikes them too; nobody has ever opened the conversation because reopening a schedule feels like reopening the contract.

The second is the utilisation trap in professional services and IT consulting. Utilisation is a necessary measure and a poor master. When it is the dominant measure, the organisation systematically underinvests in exactly the work that has no billing code — improvement, documentation, mentoring, root cause analysis. Then it wonders why the same problems recur. Pair utilisation with something that values the non-billable work, or accept that you have designed an organisation that cannot improve itself.

The question worth asking

Pick your most important operational measure. Then ask the people furthest from the boardroom, in these words:

If you wanted to make this number look good without doing better work, how would you do it?

Ask it without a notebook. The answers come quickly, and they are rarely a surprise to anyone who does the job. The surprise is only ever for the people who own the number.

Then decide — deliberately, with somebody senior in the room who owns both sides — whether what you are measuring is still a reasonable stand-in for what you want. Occasionally it is. More often it stopped being one some years ago, and the organisation has been quietly paying for the gap ever since, in a cost centre that nobody connects back to the number.

Sources

• US Senate Committee on Health, Education, Labor and Pensions, majority staff report on Amazon warehouse safety, 16 December 2024 — help.senate.gov
• Claims Journal / Bloomberg (Matt Day), Amazon ignored its own worker safety studies, Senate report says, 18 December 2024 (Project Soteria, Project Elderwand, investigation scope, Amazon’s response) — claimsjournal.com
• CNN Business, Senate report accuses Amazon of ignoring worker safety in productivity push, 16 December 2024 — cnn.com
• Senate HELP Committee, Amazon investigation materials — help.senate.gov

The findings described are those of the Senate committee’s majority staff. Amazon rejects the report, stating it is “wrong on the facts”, and that position is set out above. The case is used as a documented illustration of how measurement systems resist counter-evidence, not as a conclusion about any company’s safety record.