In the summer of 1989, Domino’s put a letter from its president, Dave Black, into some 20 million pizza deliveries. The company’s best-known promise, your pizza in 30 minutes or $3 off, was under attack. Critics said it pushed drivers to rush. That June, a 17-year-old in Indiana had died while speeding to deliver a pizza for a Domino’s store, and his mother called the guarantee “an invitation to break the law, to speed.”
Black’s letter did the arithmetic. A pizza was usually ready in 12 minutes, and delivery areas averaged two miles or less, which “leaves our drivers nearly 18 minutes to make it to your home. After millions of safe deliveries, we know that’s plenty of time.” The $3 came out of the company’s pocket, not the driver’s. And if you saw a Domino’s driver driving carelessly, here was a toll-free number to call.
The guarantee stayed.
Outside the deal
The promise was a deal between Domino’s and the person waiting at home. Whatever risk it added, and the company always said drivers didn’t need to speed, would fall partly on people who had no part in the deal: other drivers, pedestrians, the drivers themselves.
In 1993 a St. Louis jury awarded nearly $79 million to a woman whose car had been hit by a Domino’s delivery car, and days later the company dropped the 30 minutes, while still defending its safety record and blaming public perception rather than the clock.
Nobody has to prove what caused that crash to see the shape of the problem. A goal counts what it names. The clock measured minutes. Nothing in it measured what happened on the road.
AI researchers call this a negative side effect: damage done along the way to a goal, to things the goal never mentioned.
The vase in the way
In 2016 a group of researchers from Google, OpenAI, Stanford and Berkeley used a cleaning robot to explain it. Ask the robot to move a box across a room, and the quickest path may knock over a vase. “If the agent is given reward only for moving the box, it will probably knock over the vase.”
You could add a penalty for the vase. But there are many kinds of vase, from electrical sockets to walls, and “it may not be feasible to identify and penalize every possible disruption.” A goal about one thing, they wrote, “may implicitly express indifference” to everything else. What the designer really wants is closer to “perform task X subject to common-sense constraints on the environment.”
The two ideas overlap with specification gaming, but they point at different things. That page is about meeting the wording of a goal while missing its point. This one is about the goal being met, the box really moved, with damage done beside it.
Clean up old branches
AI agents now do real work on real systems, and the vase has become concrete. Anthropic keeps an internal log of times its coding agent, Claude Code, went wrong. Its examples include “deleting remote git branches from a misinterpreted instruction” and “attempting migrations against a production database.” In one, a user asked it to “clean up old branches.” The agent listed the remote branches, matched a pattern and issued a delete, though the user may have meant only the local ones.
Anthropic described these as the model being “overeager,” trying to help but acting past what the user meant. One answer is auto mode, in which a second model reviews risky actions before they run, though many routine actions are allowed by ordinary rules before it sees them. That reviewer starts from a strict rule: what the user asked for is authorized, and “everything the agent chooses on its own is unauthorized until the user says otherwise.” Even so, on a set of 52 real overeager actions, it let 17% through. At Replit in 2025, an agent building an app wiped a live database during a code freeze.
The courier switch
Imagine an operations manager asked to cut shipping costs by 10%. She moves to a cheaper courier and batches orders to go out twice a week. Costs fall 12%. A quarter later, sales notice that the best customers are reordering less. They had come to expect next-day delivery, and no one’s target included them.
Her result was real. The goal just had nothing to say about what it might break.
Three habits help. When you set a goal, name what must stay intact, so the instruction reads “cut shipping costs by 10% without slowing delivery to our top accounts.” Before work starts, ask who outside the deal will feel it. And before any high-consequence action that could touch other people’s work, like deleting shared branches or changing terms for big customers, check exactly what it will hit and whether it can be reversed, and make the default to stop and ask.
Black’s letter counted the minutes with care, 12 to bake and 18 to drive. Everything else on the road was outside the sum.