In March 2004, a fingerprint lifted from a bag of detonators in Madrid reached the FBI Laboratory in Quantico. Bombs on the city’s commuter trains had killed about 200 people. An examiner ran the print through the Bureau’s database of millions, and the computer offered 20 candidates. He settled on the fourth: Brandon Mayfield, a lawyer in Portland, Oregon.
A second examiner verified the match. A unit chief reviewed it and agreed. When the Spanish National Police laboratory reported its own result, “negativo,” the FBI lab, before finding out why the Spaniards disagreed, said it was “absolutely confident.” Mayfield was arrested on May 6.
A court appointed an independent expert. On May 19, he too said the print was Mayfield’s. That same day, Spanish police matched it to someone else, an Algerian named Ouhnane Daoud.
Four experienced examiners had looked and agreed. Count the looks again, though. After the first, every one of them began by knowing what had already been concluded.
Reading backward
The Justice Department’s Inspector General reviewed the case in 2006. Part of the cause was rare: the two men’s prints were unusually similar, with as many as 10 points in matching positions. He found no compelling evidence that the verification procedure itself had swayed the verifier.
But the first look had gone wrong in a way the later looks didn’t catch. With Mayfield’s print beside the Madrid one, the examiners reasoned “backward.” They began to “find” details in the Madrid print that were not really there, reading murky spots as points of agreement. Nothing required the examiner to write down what he saw in the Madrid print before he saw Mayfield’s, so nobody could say afterward which details had been clear to him first.
Form your view alone, from the evidence, before you see anyone else’s answer.
Soon afterward, the psychologist Itiel Dror and two colleagues showed how much expectation can matter. They gave five experienced fingerprint examiners a pair of prints each had personally declared a match about five years earlier. This time the examiners were told it was the pair the FBI had wrongly matched to Mayfield. Only one examiner still called it a match. Three called it a definite non-match, and one said there wasn’t enough to decide. Five examiners and one staged story can’t say how often this happens in real casework. They show that it can.
Washing your hands
Daniel Kahneman, Olivier Sibony and Cass Sunstein gave procedures like these a deliberately unglamorous name in their 2021 book Noise: decision hygiene. Kahneman later explained the choice. A remedy for a known bias is like a vaccine, aimed at one disease. “You wash your hands to kill germs, but you have no idea what germs you’re killing and if you’re lucky, you will never know.”
Among the parts he listed, the first was independence: judgments, and evaluations of different aspects of a problem, made separately. The second was delaying intuition, because an early impression steers what you go looking for.
The Inspector General’s recommendations read like the same list in a lab’s vocabulary. Record what you see in the crime-scene print before the comparison begins. Let verifiers work without knowing whether anyone has already declared a match. Neither rule names the error it prevents.
Hygiene costs time; Kahneman said to reserve it for decisions that deserve the effort. And it has limits: a blind second look at an unusually similar print might still have gone wrong.
The answer in the first message
Imagine you’re on call, and a customer has been charged twice. Before you open anything, a teammate messages you: “Payment provider timeout again, same as March.” You open the logs and there it is, a timeout, right before the second charge. You close the ticket.
A week later, the double charge happens again with no timeout anywhere. Reading back, you find the retry button that let the customer pay a second time, a few lines above the timeout you had been told to look for.
The other order costs a few minutes. Open the logs before the chat, note what they show, then read the theory. The retry button might have made it onto that list.
A judge that reads the answer first
Language models that grade other models’ work have their own version of the verifier’s problem.
In a 2023 study, researchers at Peking University, Tencent and the University of Hong Kong asked ChatGPT to compare answers from two chatbots, Vicuna-13B and ChatGPT itself, on 80 questions. The prompt asked for scores first and reasons second, and told the judge not to let the order of the answers affect its judgment. Vicuna won 2.5 percent of the comparisons when its answer came first and 82.5 percent when it came second. On 66 of the 80 questions, swapping the order changed the verdict. GPT-4 as judge did better, changing on 37.
The fixes were hygiene. Ask for the reasons before the score, sampled three times. Then score both orders and average. Together, these raised ChatGPT’s agreement with the majority of three authors’ own ratings from 44.4 to 58.7 percent. The study changed several things at once, so it doesn’t show which one mattered most.
If you use a model to pick between two drafts, run it both ways, A first and then B first, and ask for the reasons before the score. A verdict that flips with the order is a lead to look into, not a ranking. Repeat a few times, then read the two drafts yourself against what the task actually asked for. And when you want a second model to check the first one’s verdict, give it the drafts, not the verdict.
The FBI’s own fresh look came after Spain had named Daoud. Overnight on May 23, a unit chief brought in another unit chief to re-examine it. One reason he gave the Inspector General: that colleague had not been involved in the identification.
