Home › Stories
Tesla · 2018 · An AI story

The Gore on Highway 101

Twice in the month before, a Tesla on Autopilot had steered toward what appears to have been the same highway split, and both times its driver corrected it within two seconds. The third time, he didn't.

◷ 2 min read

At 9:27 on the morning of March 23, 2018, a Tesla Model X was heading south on US Highway 101 in Mountain View, California, on the driver’s commute. Autopilot was on. It was steering the car within its lane and keeping pace with the car ahead, with the cruise speed set to 75 mph.

About six seconds from the end, Autopilot’s lane-keeping system steered left, into the gore, the paved wedge of no-man’s-land where an exit ramp splits from the main lanes. The car ahead carried on in its lane. With no vehicle in front of it any more, the Tesla sped up toward its set speed. Its collision systems did not detect the crash barrier at the end of the gore. There was no warning and no automatic braking.

The car hit the barrier at about 71 mph. The barrier had been damaged in an earlier crash and not repaired. The driver died of his injuries.

The National Transportation Safety Board reconstructed his last minutes from the car’s data and his phone. His hands were not detected on the steering wheel, and he made no attempt to steer or brake. A game was the frontmost app on his phone, with data use consistent with play.

Investigators also went back through a month of his morning drives. Twice, on February 27 and March 19, Autosteer had made a similar move to the left, apparently near the same split. Both times the driver had steered back within two seconds.

Autopilot was what engineers call a Level 2 system: “partial driving automation.” It controlled steering and speed, but under that definition the driver remained responsible for watching the road, understanding the system’s limits, and taking over at any moment. The car judged whether he was doing that by sensing his hands on the wheel through steering torque.

The NTSB’s probable cause named both halves. Autopilot had steered into the gore “due to system limitations,” and the driver had failed to respond, likely distracted, and with “overreliance” on the system. Contributing was “the Tesla vehicle’s ineffective monitoring of driver engagement.” The board concluded that steering torque was “an ineffective surrogate measure” of whether a driver is paying attention, and warned that unless Tesla limited Autopilot to the conditions it was designed for, use beyond them was “foreseeable.”

What this story shows

Levels of AutomationAutopilot steered and held speed, the jobs of a Level 2 system. Watching the road and catching its mistakes stayed with the driver, and the car's way of checking that he was doing so didn't work.Human-on-the-Loop (Supervisory Control)The car steered into the gore with about six seconds to go. The NTSB found the driver likely distracted by a phone game, and the car's monitoring of his attention ineffective.Trust CalibrationThe driver had overridden Autosteer twice that month, apparently near this same split. The NTSB still named his "overreliance" on the system as part of the cause.

Sources

  1. NTSB, Collision Between a Sport Utility Vehicle Operating With Partial Driving Automation and a Crash Attenuator, Mountain View, California, March 23, 2018 (HAR-20/01), 2020