Home › Stories
U.S. Army and Royal Air Force · 2003

One Minute to Decide

A Patriot missile battery's computer classified a returning British jet as an incoming missile. The crew had about a minute to decide whether to fire on that call. They had been trained to trust the system.

◷ 2 min read

In the early hours of March 23, 2003, local time, in the first days of the Iraq war, a Royal Air Force Tornado was heading home to its base in Kuwait. The pilot had just started the descent. The navigator had checked the switches for the jet’s identification system, which answers a friendly radar’s challenge with a coded reply.

Nearby, a U.S. Army Patriot battery was watching for Iraqi ballistic missiles. Its radar picked up the Tornado. The symbol on the crew’s screen said an anti-radiation missile, a weapon that homes in on radar, was coming straight at them. The system challenged the track for an identification reply. There was none. The track met every criterion for a hostile missile.

The crew fired. Both men in the Tornado were killed instantly.

Britain’s Board of Inquiry, working alongside the U.S. Army’s investigation, found no single cause. The Tornado’s identification system had a fault the crew didn’t know about, most likely a power failure the cockpit didn’t display. The Patriot’s criteria for recognizing an anti-radiation missile had been written for every such missile in the world, so they were very broad; the Tornado’s descent fit them. The battery’s own communications gear was still in transit from the United States, so it lacked the wider picture of the airspace that might have marked the track as friendly.

And there was the crew. “Patriot crews are trained to react quickly, engage early and to trust the Patriot system,” the summary said. They had about one minute to decide. If they had waited, the board thought, the system would probably have reclassified the Tornado as its flight path changed. Their training had focused on recognizing threats, not on spotting false alarms.

Later in the war, a Patriot also shot down a U.S. Navy F-18.

John Hawley, an Army engineering psychologist with more than 35 years on Patriot and other Army air defense systems, led the Army’s efforts afterward to improve operators’ vigilance. He wrote that the Army had “trusted the system in a naïve manner” and had not faced the fact that supervising a nearly autonomous system is a different job from controlling it by hand. He also faulted the design: Patriot’s automatic mode was “mostly an all-or-none operating mode,” with few “decision leverage points” where operators could influence its engagement logic. His hardest lesson: “an automated system in the hands of an inadequately trained crew is a de facto fully automated system.”

The Army made changes. Training now included incidents like these, and crews were taught to query the system to confirm or challenge what it said a track was. The most important change, Hawley wrote, was “acceptance of the fact that the system is not always right.”

What this story shows

Human-on-the-Loop (Supervisory Control)The decision to fire was the crew's, but they had about a minute, no wider picture of the airspace, and training that stressed reacting fast and trusting the machine.Trust CalibrationPatriot crews were trained to "trust the Patriot system," and their training focused on real threats rather than on spotting false alarms.Levels of AutomationHawley called Patriot's automatic mode "mostly an all-or-none operating mode," with few points where operators could shape what it did.Ironies of AutomationThe battery was watching for ballistic missiles, the job its automation was built for. The aircraft it shot down fell into the gray area left to the humans.

Sources

  1. Ministry of Defence, Military Aircraft Accident Summary, Aircraft Accident to Royal Air Force Tornado GR Mk4A ZG710, May 2004
  2. John K. Hawley, Patriot Wars, Automation and the Patriot Air and Missile Defense System, Center for a New American Security, January 2017