On a Friday morning in March 2018, a Tesla Model X was carrying its driver south on Highway 101 toward work in Silicon Valley. Autopilot was steering and keeping pace with the car ahead.
About six seconds before the end, the steering system turned left, into the gore, the paved wedge where an exit ramp peels away from the main lanes. The car ahead went straight. With the road in front now clear as far as it could tell, the Tesla accelerated toward its set speed of 75 mph.
It had done something similar twice in the past month, near what appears to have been the same split. Both times, the driver had steered back within two seconds.
This time, investigators found, his hands were not on the wheel. A game was open on his phone.
Not a switch
Tesla’s Autopilot was what the auto industry calls Level 2, “partial driving automation.” The scale, set by the engineering body SAE International, has six steps, from Level 0, “no driving automation,” to Level 5, “full driving automation.” Autopilot’s cruise control and its steering assist each counted as Level 1 on their own; together they made Level 2. At Level 2, as the National Transportation Safety Board summarized it, the car controls steering and speed, while it remains “the driver’s responsibility to monitor the automation, maintain situational awareness of traffic conditions, understand the limitations of the automation, and be available to intervene and take full control of the vehicle at all times.”
That one label hides a split job. The car did the acting. The noticing, at least of anything unusual, was still the driver’s. And the car’s way of checking that he was noticing, sensing his hands on the wheel through steering torque, was in the NTSB’s words “an ineffective surrogate measure.”
Human factors researchers had described this problem long before. In 2000, Raja Parasuraman, Thomas Sheridan and Christopher Wickens proposed that automation applies to four kinds of function: taking in information, analyzing it, choosing what to do, and doing it. Each can be automated “across a continuum of levels,” from fully manual to fully automatic, and “a particular system can involve automation of all four types at different levels.” They also warned that automation “does not merely supplant but changes human activity.”
Seen that way, the Tesla was highly automated at doing, partly automated at noticing, and left the hardest piece, spotting what its sensors missed, to a person whose attention it could not reliably check.
The payables desk
The same thinking works for an office process. Imagine an accounts payable clerk who brings in software to handle supplier invoices. She doesn’t pick one setting for the whole thing. Reading invoices and pulling out the amounts, she lets the software do fully. Matching each invoice to a purchase order, the software does and flags mismatches for her. Deciding to pay, it proposes and she approves, for new suppliers and anything over a set amount. Actually sending money to a regular supplier for a small, matched invoice, it does on its own and reports afterward.
Four parts of one job, four different levels, each chosen by asking two questions: how reliable is the machine at this part, and what does a mistake here cost? The parts left to her are ones she can actually perform: the approvals arrive one by one, with the information she needs, before the money moves.
Permissions for an agent
Anthropic’s coding agent, Claude Code, offers a ladder of permission modes. In Manual mode it runs only reads without asking and stops before most file edits, shell commands and network access. An accept-edits mode also lets file edits and common file commands through. In auto mode, a second model reviews actions in place of the user. A bypass mode lets everything run, and the documentation says it is for “isolated containers and VMs only.” On top of any mode, users can write rules that allow, ask about or deny specific tools, and deny rules block in every mode.
Permissions are one way to draw the line on execution: what the agent may do without a person. They don’t settle the other parts of the job. Whether its analysis is sound and its recommendations are good still has to be judged separately. The useful habit is to set the line deliberately: high automation where mistakes are cheap and easy to undo, a hard stop where they aren’t, and for whatever is left to a person, a check they can realistically make.
The NTSB found that the Tesla had no way to make sure its driver was still doing his part. Twice that month, he had been.