On a spring day in 1980, five of the most sought-after high school prospects in Southern California lined up along the left field foul line at Herbert Hoover High in San Diego. Pat Gillick, then general manager of the Toronto Blue Jays, stood in the outfield with a stopwatch. Five or six scouts around him held theirs.
Sixty yards. Billy Beane finished three full strides ahead of the next boy. Gillick’s watch read 6.4 seconds. In Michael Lewis’s Moneyball, Gillick had the distance paced off again and sent the boys back to the line. “Gillick’s job is to believe what he sees and disbelieve what he doesn’t and yet he cannot bring himself to believe what he’s just seen,” Lewis writes.
The scouts carried checklists of five “tools”: the abilities to run, throw, field, hit, and hit with power. They also looked at Beane’s face, and he had what they called the Good Face. “I never looked at a single statistic of Billy’s,” one of them admitted, in a passage from the book quoted by the economist Richard Thaler and the law professor Cass Sunstein. “It couldn’t have crossed my mind.”
Beane went in the draft’s first round. As a major leaguer, he never became what they had pictured.
A lot you couldn’t see
He hit .219 in 301 at-bats over six seasons, with 80 strikeouts and 11 walks.
The scouts checked everything the field could show. Nothing on it said what it couldn’t. Lewis’s own diagnosis, as Thaler and Sunstein summed it up, was that scouts trusted their own eyes, and there was “a lot you couldn’t see when you watched a baseball game.”
We build a confident story from what’s in front of us and rarely ask what’s missing.
Daniel Kahneman named the habit WYSIATI: what you see is all there is. In a 2011 Harvard Business Review article, he, Dan Lovallo and Olivier Sibony wrote: “Because our intuitive mind constructs a coherent narrative based on the evidence we have, making up for holes in it, we tend to overlook what is missing.” He used it again that year in Thinking, Fast and Slow.
A photo cropped at the edges still looks like a whole photo. Nothing in the frame tells you someone was standing just outside it.
Told that half was missing
Surely knowing that a piece is missing helps. In 1996, Lyle Brenner, Derek Koehler and Amos Tversky tested that. Students at San Jose State and Stanford read short legal cases. Some read both sides and voted, as a jury. Others got the background and one side’s arguments, and were told repeatedly, in writing and out loud, that the jury had read both.
The one-sided readers predicted how many of 20 jurors had found for the plaintiff. Their guesses leaned toward the side they had read. Their 90 percent ranges were also narrower, a sign of more confidence, though they were no more accurate, and in some comparisons less. The confidence gap was small: in the first experiment, ranges averaged 10.4 votes wide against the jury’s 11.1. The lean was strongest when the sides were evenly matched.
One step helped a lot. Students who first rated whether they expected the unseen side to be weaker or stronger leaned far less: the gap between plaintiff-only and defendant-only predictions averaged 2.7 votes out of 20, against 7.1 for students who skipped that step.
What the numbers left out
In 2003 Thaler and Sunstein reviewed Moneyball in The New Republic and, as Lewis later recalled, traced the scouts’ misjudgments to workings of the mind that Kahneman and Tversky had described years before. Until then, Lewis wrote, he didn’t believe he’d heard of either man.
Beane’s front office looked instead at records of what a hitter had done, such as how often he got on base. Records have edges too. The book left in the background the A’s three excellent pitchers, Tim Hudson, Mark Mulder and Barry Zito; Lewis said later that everyone already knew how good they were. “I never meant to say that experience is completely irrelevant,” he said in 2022.
The proposal with no other side
Imagine a colleague asks you to approve moving the team to a new ticketing system. The deck is good: three pain points, a cheaper vendor, a pilot team that loves it. Nobody was asked to make the case for staying.
The migration weekend and the people outside the pilot appear on no slide, so they weigh nothing.
Borrow the study’s step. Before you decide, write down how strong you’d expect the case for staying to be, and what it would say. Kahneman, Lovallo and Sibony also suggest a question from the Harvard Business School professor Max Bazerman: “If you had to make this decision again in a year, what information would you want, and can you get more of it now?” Often, they note, the data are unavailable. Sometimes they aren’t.
When the documents run out
Language models handed a stack of documents can do something similar. In a study published at ICLR 2025, Hailey Joren and colleagues gave models questions along with retrieved text, and had an automated rater judge whether the text held enough to answer. Gemini 1.5 Pro, GPT 4o and Claude 3.5 did well when it did, but “often output incorrect answers instead of abstaining when the context is not” sufficient.
That was on question-answering benchmarks, with 2024 models and an AI grader; newer models may differ. The models also answered correctly 35 to 62 percent of the time when the text fell short, so partial evidence wasn’t useless.
Pick a question your documents answer, remove the one that holds the answer, and ask again in fresh chats a few times. If the model answers anyway, check whether it’s right, ask for the source of each claim, and open those sources.
If the answer comes back as fluent as before, the only thing that changed is the page you took away.