Home › Stories
Uber · 2018 · An AI story

The Backup Driver

The self-driving car saw her 5.6 seconds before impact. The human who was supposed to step in was looking at her phone.

◷ 2 min read

On the night of March 18, 2018, a Volvo SUV was driving itself along a test route in Tempe, Arizona. It belonged to Uber, which was developing self-driving cars, and it had been in autonomous mode for about 19 minutes. In the driver’s seat sat a vehicle operator. Her job was to watch the road and take over if anything went wrong.

At 45 miles an hour, the car approached a woman pushing a bicycle across the road, away from any crosswalk.

The car’s system detected her 5.6 seconds before impact. It kept tracking her, but it never correctly worked out that she was a pedestrian or where she was going. By the time it decided a collision was coming, it was too late for its own braking to avoid one. And it had been designed not to slam on the brakes by itself in that situation. It relied on the human to do that.

The human was looking down toward the bottom of the center console, where she had put her phone. She looked up about one second before impact. She began to steer 0.02 seconds before the car hit the woman, who died at the scene.

The National Transportation Safety Board pieced together what had happened in the car. While it was moving, the operator spent 34 percent of the time looking down toward the console. In the last three minutes she glanced there 23 times. Records from the streaming service showed her account playing a TV show for the entire trip.

The NTSB’s probable cause put her distraction first. But the board did not stop there. It also faulted Uber’s safety culture, including “ineffective oversight of vehicle operators” and a “lack of adequate mechanisms for addressing operators’ automation complacency.”

The criminal case went the other way. In 2019 prosecutors found “no basis for criminal liability for the Uber corporation.” In 2020 the operator was charged with negligent homicide, and in 2023 she pleaded guilty to endangerment and was sentenced to three years of supervised probation.

There was one more detail. Until about five months before the crash, Uber’s test cars had carried two operators, one to watch the road and one to log what the system was doing. Uber had merged the two jobs into one. The second person, the NTSB noted, had also been a reminder to the first of what her job was.

After the crash, Uber went back to two.

What this story shows

Ironies of AutomationThe car needed its human only in rare emergencies, which meant long, quiet watching, the kind vigilance research says people struggle to sustain.Human-on-the-Loop (Supervisory Control)The design relied on the operator to brake in an emergency, and the NTSB found Uber had no adequate way to keep operators attentive.Automation BiasInvestigators named "automation complacency" as a risk Uber had no adequate way to deal with.Moral Crumple ZoneProsecutors found no basis to charge Uber. The operator, left alone in a job the company had cut from two people to one, pleaded guilty.

Sources

  1. NTSB, Collision Between Vehicle Controlled by Developmental Automated Driving System and Pedestrian, Tempe, Arizona, March 18, 2018 (HAR-19/03), 2019
  2. BBC News, Uber 'not criminally liable' for self-driving death, March 6, 2019
  3. BBC News, Uber's self-driving operator charged over fatal crash, September 16, 2020
  4. Associated Press via Transport Topics, Backup Driver in Autonomous Uber Death Pleads Guilty, July 28, 2023