In October 2002, Steve Ballmer, Microsoft’s chief executive, wrote to customers about something most of them never saw: what happened after a program crashed. People hardly ever phoned support about a crash; “instead,” he wrote, “they close the program.” The bug vanished with the window, and Microsoft never heard about it.
That was a real cost. Even a fairly simple product, Ballmer wrote, had millions of lines of code, with “many places for bugs to hide.” Testing had not found them all. So which ones should Microsoft fix first?
A small team in the Office group had built a way to find out. When something went wrong, the software offered to send Microsoft an error report. Ballmer admitted there were risks “in offering this option to have software ‘phone home’ like E.T.” Customers sent them, and the reports landed on a cluster of dedicated Microsoft servers.
If the bugs had been spread evenly, every fix would have bought the same small drop in crashes. Ballmer reported something else. “About 20 percent of the bugs cause 80 percent of all errors,” he wrote. Then he added that this was “stunning to me”: “one percent of bugs cause half of all errors.”
One big hole, twenty pinholes
A few causes often make most of the trouble. Count before you fix.
A bucket has one big hole and twenty pinholes. Patch the pinholes nearest your hand first and you can work all afternoon while most of the water keeps pouring out. Find the big hole first and one patch changes what the bucket can hold.
Some causes simply get more chances to strike. A bug in code that runs every time a program starts gets more chances to crash it than one hiding in a menu almost nobody opens. From inside the bug list, both look like one line. Only counting tells them apart.
Ballmer’s letter gave the payoff. With Office XP Service Pack 2, he wrote, error reports helped Microsoft eliminate more than half of all Office XP errors.
The man who named the pattern later confessed the name was wrong. Joseph Juran, a quality engineer, noticed in the mid-1920s that a few kinds of defects accounted for most of the defectiveness. Later, a General Motors manager showed him how closely GM’s executive salaries fit a model built by the Italian economist Vilfredo Pareto, who had studied how unequally wealth was spread. Writing his Quality Control Handbook (1951), Juran needed a short label for the general rule and borrowed Pareto’s name. He also coined “the vital few and trivial many,” later softened to “useful many.” In 1974 he admitted that Pareto’s models were about economics and were not meant for other fields. “Had I been structured along different lines,” he wrote, “assuredly I would have called it the Juran principle.”
And 80/20 is a shape, not a constant. Juran’s own definition has no numbers in it: “a relative few of the contributors account for the bulk of the effect.” Ballmer’s letter shows it too. The same pile of reports gave him one split at 20 and 80, and another at 1 and half.
The failures that don’t crash
Microsoft’s system, later called Windows Error Reporting, sorted incoming reports into “buckets,” each meant to hold the reports likely caused by one bug. Every crash the system collected, and the customer agreed to send, landed in a bucket.
When an AI assistant gets a date wrong, nothing crashes. The conversation carries on and no report goes anywhere. Someone has to read it and decide what went wrong.
Nurture Boss did that with its AI assistant for apartment rentals, according to Hamel Husain, who worked with the team and wrote it up in March 2025. They put conversations in a spreadsheet, one per row, with a short note on each bad response. They used an LLM to group the notes into failure types, then counted. Three issues accounted for over 60 percent of all problems: conversation flow (missing context, awkward responses), handoffs (not recognizing when to pass a user to a human), and rescheduling. Dates were a sore spot, with requests like “let’s schedule a tour two weeks from now.” After the team built tests for exactly that, the success rate on date handling went from 33 percent to 95.
You can run the same count on an AI tool you rely on. Pull fifty conversations where it let you down, give each a single short label, and sort. Change one thing aimed at the top label, then count again next week and see whether that row shrank.
The complaint pile
Imagine you look after the customer inbox for a small online shop, and 212 complaints have piled up this week. Last time this happened you started at the top and answered them one by one.
This time you spend an hour giving each email one tag. Four of the tags circle the same thing: no confirmation email, no tracking number, “did my order go through?”, and people who ordered twice because they weren’t sure. Together they cover more than half the pile. They all lead to one place. Since the checkout page was updated, order confirmations, which carry the tracking link, have stopped going out. The rest of the pile, wrong sizes and damaged boxes, has causes of its own.
The tags could have come out nearly even. Juran called a chart with no clear break the “Awkward Zone.” When that happens, ask whether any tag points at something you could change. If none does, this pile may have no small set of causes, and answering it one by one is the job.
Here, one does. First you write to everyone who ordered since the update: the order went through, here is the tracking link, here is how to cancel a duplicate. Then you get the confirmation email working again, and answer the sizes and the damaged boxes. By the next week, “where is my order?” has gone from the biggest tag in the inbox to a few emails a day.