Gen. Buck Kernan made the enemy a promise. Millennium Challenge 2002, the $250 million war game his US Joint Forces Command ran that summer, was meant to put a new concept through a hard test before it became doctrine. Kernan called it “free play” and said the enemy had “the opportunity to win.”
The enemy was Paul Van Riper, 64, a retired Marine lieutenant general. He expected the Americans to strike first, “so I decided I would attack first.”
As a US carrier battle group sailed into the Persian Gulf, small boats and propeller planes, many of them civilian, circled aimlessly around it. Then the signal came, not by radio but, as the Guardian told it weeks later, as a coded message from the minarets at the call to prayer. Missiles flew from commercial ships. By the Washington Post’s count, 19 American ships were destroyed within ten minutes, the aircraft carrier among them.
None of it was real. Which meant it could be taken back.
Refloat the fleet
“A phrase I heard over and over was: ‘That would never have happened,’” Van Riper told the Guardian. An adjudicator ruled exactly that, and all but four of the ships were refloated.
He was then barred from using chemical weapons, as not fitting his character’s mindset. On 29 July, the exercise director began passing guidance to his chief of staff, over his head. “I stayed on to give advice, but I stopped giving orders,” Van Riper said. On 15 August, the game ended in an American victory.
The Pentagon had a point: Gen. Peter Pace argued that bringing the dead back to life got 13 more days of experiment out of the game. Van Riper’s complaint was that this game had been sold as free play, to test whether the concepts were sound. His report concluded that the result “was preordained from at least 31 July.”
The command had found a skilled opponent and told him to try to win. It never settled what his winning would be allowed to change.
Appoint someone to argue against the plan, then let the argument change something.
The officer paid to object
Before the Catholic Church made anyone a saint, an official called the Promoter of the Faith had a duty, in the words of the Catholic Encyclopedia of 1907, “to prepare in writing all possible arguments, even at times seemingly slight, against” it. His popular title was advocatus diaboli, the devil’s advocate.
A group that likes its plan tends to hear objections as disloyalty. Hand one person the job of objecting, and the objection becomes a task; nobody has to volunteer to be the one who isn’t on board.
That fixes who speaks. It does nothing about who listens.
The trouble with pretending
It can fail a second way, which the psychologist Charlan Nemeth has studied. A 2001 paper by her team compared a real dissenter with three versions of a devil’s advocate, scoring how many solutions people produced and how good they were. The authentic minority beat all three. An earlier study, as the paper summed it up, had found that a devil’s advocate seemed mostly to prompt people to shore up the view they already held.
Her explanation, in a 2018 interview, is about belief: with a role-player, “you can’t argue with the person who’s pretending.” The part is weaker than the real thing, not useless. Van Riper was playing to win. The game, he told the Guardian, “was described as free play,” with “two sides trying to win.” Pretending wasn’t his problem. Winning was.
An objection with nowhere to land
Imagine your support team moves to a new ticketing system next month. At the final review you ask your most careful analyst to argue against it. She does: the new system can’t import the old tickets, so agents will lose every customer’s history of complaints.
The answers come fast: customers don’t expect us to remember; we’ll export a spreadsheet. Nobody checks either one. She stops, and the minutes record no objections.
Change one thing. Before she starts, the group writes down what she would have to show for the date to move: if one ticket in ten in a normal week refers to an earlier problem, the old system stays open, read-only. She pulls a week of tickets and counts. Now her objection can win, the way Kernan said the enemy could.
A devil’s advocate made of software
In 2024, Chun-Wei Chiang and three colleagues at Purdue University gave the job to a language model. Online, 350 people, in small groups that started as threes, judged whether criminal defendants would reoffend within two years. Each group saw the recommendation of an AI model the researchers had trained on a deliberately biased sample so it did badly on one kind of defendant.
Groups were randomly assigned no devil’s advocate or one of four built on GPT-3.5-turbo, which argued against either the AI’s recommendation or the group’s majority, and either spoke once at the start or kept joining in.
The version the authors found significantly raised accuracy, against groups with none, was the one that argued against the AI and kept arguing. The gain was small. Where the AI was wrong, none of the four significantly changed how often groups went along with it. The authors caution that these were anonymous strangers, meeting once, without expertise, in one domain.
You can aim one at your own AI’s answers. In a fresh session, paste the facts and the recommendation and ask for the strongest case against it. Run it several times; one sharp point from one run is a lead, not a finding. Check each against the facts. First, write down what an objection would have to show to change your mind.
Expect it to sting. People whose advocate kept arguing against the AI rated themselves less successful than those whose advocate argued against it once and went quiet.
In Norfolk, the refloated fleet won.