Home › The AI Era › Human-on-the-Loop (Supervisory Control)
The AI Era · Human in the Loop

Human-on-the-Loop (Supervisory Control)

Let the machine act on its own while a person watches and can step in, if they truly can.

◷ 4 min read▤ 3 true storiesWorks the same with AISee its neighbors on the map →

In the early hours of March 23, 2003, a U.S. Army Patriot battery in Kuwait was watching the sky for Iraqi ballistic missiles. A track appeared on the crew’s screen. The system classified it as an anti-radiation missile, a weapon that homes in on radar, heading straight for them. It challenged the track for a friendly identification code and got no answer.

The machine had made its call. The decision to fire was still the crew’s, and they had about a minute.

They fired. The track was a Royal Air Force Tornado on its way home, with a faulty identification system that its two crewmen didn’t know about. Both were killed.

The Patriot crew had followed their rules and their training. Britain’s accident summary spelled out what that training was: “Patriot crews are trained to react quickly, engage early and to trust the Patriot system.”

Watching instead of doing

Engineers call the crew’s role supervisory control, or being “on the loop.” John Hawley, an Army engineering psychologist who spent more than 35 years with Patriot and other air defense systems, defined it plainly: the operator “sets goals, monitors system actions, and intervenes when necessary.” It differs from being “in the loop,” where nothing happens until a person approves each step. On the loop, the machine acts by default and the person’s job is to catch what it gets wrong.

The Patriot crew that night sat between the two. The machine detected and classified the track on its own, and the crew watched that work, while the decision to launch still needed them. Both jobs, supervising the classification and approving the shot, depended on the crew being able to tell when the machine was wrong.

Catching those errors is the hard part. On paper, the decision to fire belonged to the crew. In practice, the Board of Inquiry found, they had a minute, no wider view of the airspace because their communications equipment hadn’t arrived, and training focused on recognizing threats rather than false alarms. Had they waited, the board thought, the system would probably have reclassified the jet as its flight path changed.

Hawley drew the conclusion bluntly: “an automated system in the hands of an inadequately trained crew is a de facto fully automated system.” A person who can’t realistically intervene isn’t supervising. They’re just present.

Managing by exception

Most managers have made the same move without the jargon. Imagine a customer service lead who used to approve every refund herself. As the team grew, she set a policy: agents approve refunds under a limit on their own, the refunds wait in a queue until the evening payment run, and she reviews the day’s list before it goes out. She now spends less time per refund and catches more of the strange ones, like the same customer refunded by three different agents in a week.

The arrangement works because of three things the Patriot crew lacked. She sees the list before the money leaves. She knows what normal looks like, because she spent months approving refunds one by one. And she has a view the agents don’t: the whole team’s activity in one place, which lets her check each refund against the rest. Take any one away, say the list arrives a month late, and she is back to being present rather than in charge.

Supervising an agent

Anthropic’s 2026 study of Claude Code, its coding agent, shows users making the same move. At the time of the study, Anthropic wrote, its default settings required users to approve each action manually. Newer users, with fewer than 50 sessions, let it run with full auto-approve in roughly 20% of sessions; by 750 sessions, over 40%. Yet experienced users also interrupted more: in around 9% of turns, against 5% for new users.

Anthropic read this as a change of strategy, from checking every action before it happens to letting the agent work and stepping in “when something goes wrong or needs redirection.” The authors noted that part of the shift may simply reflect people changing those settings, and experienced users may also be taking on harder tasks.

The study found a second form of oversight that comes from the agent itself. On the most complex tasks, Claude Code stopped to ask for clarification more than twice as often as humans interrupted it. The authors warned against reading too much into this, since it may not stop at the right moments. Still, an agent that pauses when unsure carries part of the supervision.

The Patriot lessons translate directly. Give the person watching an agent enough time to intervene, which means stopping points before actions that can’t be undone. Give them a way to check the agent’s work independently, such as the customer’s history or a log of what changed in production this morning. And give them practice at the agent’s false alarms, not only its successes. After 2003, Hawley wrote, the Army’s most important change was “acceptance of the fact that the system is not always right.” Crews were taught to question what the machine told them a track was.

On paper, the Patriot crew had a minute and the final say. To use either, they would have needed the wider picture their equipment couldn’t give them and the practice at doubting the machine their training didn’t. Training alone would not have been enough.

Stories behind it

Play it

Sources

  1. Ministry of Defence, Military Aircraft Accident Summary, Aircraft Accident to Royal Air Force Tornado GR Mk4A ZG710, May 2004
  2. John K. Hawley, Patriot Wars, Automation and the Patriot Air and Missile Defense System, Center for a New American Security, January 2017
  3. Anthropic, Measuring AI agent autonomy in practice, February 18, 2026
  4. NTSB, Collision Between Vehicle Controlled by Developmental Automated Driving System and Pedestrian, Tempe, Arizona, March 18, 2018 (HAR-19/03), 2019
  5. BBC News, Uber 'not criminally liable' for self-driving death, March 6, 2019
  6. BBC News, Uber's self-driving operator charged over fatal crash, September 16, 2020
  7. Associated Press via Transport Topics, Backup Driver in Autonomous Uber Death Pleads Guilty, July 28, 2023
  8. NTSB, Collision Between a Sport Utility Vehicle Operating With Partial Driving Automation and a Crash Attenuator, Mountain View, California, March 23, 2018 (HAR-20/01), 2020