On the morning of November 12, 2001, American Airlines Flight 587 climbed out of New York’s Kennedy airport behind a Japan Air Lines 747 and flew into its wake for the second time. The bump was mild. The National Transportation Safety Board later found it “did not place flight 587 in an upset condition.”
The first officer reacted hard. In under seven seconds he pushed the rudder pedals right, left, right, further right, left and right again. Each push swung the airplane’s nose sideways, and each sideways swing loaded the tail fin. The captain said, “Hang onto it. Hang onto it.” At 9:15 and 58 seconds a fitting holding the fin broke, and the fin came off. Everyone aboard, and five people on the ground, died.
The board’s report contains a hard sentence. If the first officer had stopped “at any time” before the fin broke away, “the natural stability of the airplane would have returned the sideslip angle to near 0º, and the accident would have been avoided.” The airplane had been settling itself. The corrections kept it from settling. The NTSB also blamed a rudder system that was unusually sensitive at high speed, and training that may have reinforced his tendency to respond aggressively to wake turbulence. The case has the rest.
The funnel
The quality pioneer W. Edwards Deming had a name for correcting a system that would have been fine if left alone. He called it tampering, and he showed it with a funnel.
Hold a funnel over a target on a sheet of paper and drop a marble through it. The marble won’t land on the target every time. It bounces a little this way or that, for reasons nobody controls. The natural response is to move the funnel to make up for the last miss. Landed two inches left? Shift the funnel two inches right.
That feels like good management. It makes things worse. Deming set out four rules for moving the funnel. In the version summarized by the quality engineer Thomas Pyzdek, leaving the funnel alone, rule 1, keeps the marbles’ spread as small as it can be. Moving it to cancel out the last miss, rule 2, keeps them centered but doubles the variance. Other rules send the drops drifting off the paper. The marble’s bounce was random. Correcting for it just added each correction on top of the next random bounce.
The Deming Institute lists everyday versions: adjusting a process whenever one result falls out of range, changing policy after every employee survey, resetting a quota to match last month’s output, setting budgets from last year’s variances.
When to touch the funnel
None of this means never adjusting. Some changes really are signals: a broken tool, a new supplier, a real shift. Deming’s tool for telling the two apart was the control chart, which shows the range a stable process normally wanders within, so a point outside it stands out. That distinction has its own page, common and special causes. Tampering is what happens when every point is treated as special.
The other choice is to change the system itself, on purpose and on evidence, instead of nudging it after each result. Chasing the last marble doesn’t change the system. It only adds one more source of wobble.
The weekly rota
Imagine the manager of a small café who sets next week’s staff rota from this week’s takings. A quiet Tuesday, so next Tuesday gets one fewer person. A busy Saturday, so one more. But Tuesdays and Saturdays bounce around on their own with the weather and local events. The manager might end up short-staffed on the very days that bounce back, and overstaffed on the ones that dip, with the takings swinging more than before. Looking at three months of Tuesdays before changing anything might show there was nothing to correct.
Reactive loops
People building AI agents meet the same trap, because an agent’s behavior varies from one run to the next. “A task that passed on one eval run might fail on the next,” Anthropic wrote in a guide to evaluating agents. Without a proper set of tests, it said, debugging becomes reactive: “wait for complaints, reproduce manually, fix the bug, and hope nothing else regressed.” Teams like that “get bogged down in reactive loops,” fixing one failure, creating another, “unable to distinguish real regressions from noise.”
Its advice is Deming’s in a new setting. Run each test several times and measure how often the agent succeeds, instead of reacting to one failed run. That helps tell a real regression from a bad run before anyone changes the prompt or the tools.
On Flight 587, the bump had already passed. The first officer kept answering it anyway.