The Unavoidable Folly of Making Humans Train Self-Driving CarsUPDATE: On Thursday, May 24, the National Transportation Safety Board released its preliminary report from its investigation into Uber's fatal self-driving car crash in March in Tempe, Arizona.
The report explains that although the car's software determined it needed to stop to avoid hitting the woman crossing the street, it wasn't programmed to make emergency braking maneuvers. Instead of risking “erratic vehicle behavior” by letting the car decide when to slam on the brakes, Uber relies on the human behind the wheel to take control when necessary.
Problem is, Uber's safety driver wasn't looking at the road in the moments leading up to the crash. This story about why humans are ill-suited to this kind of work originally ran on March 24, 2018. We've updated it to include new details from the report.
So they are blaming the driver when the car's programmers failed to program the car to stop.
Isn't it the point of a self driving car to prevent an accident? What is the point of programming a self driving car to not execute a potentially life-threating accident?
The National Transportation Safety Board's preliminary report from its investigation into the crash reveals that the car's sensors detected Herzberg about six seconds before the crash, and that the software classified her as an unknown object, then as a vehicle, and finally as a pedestrian.
Less than two seconds before impact, the car determined it needed to stop. But it couldn't. "According to Uber, emergency braking maneuvers are not enabled while the vehicle is under computer control, to reduce the potential for erratic vehicle behavior," the report says. "The vehicle operator is relied on to intervene and take action. The system is not designed to alert the operator." But video shows that the car's operator, Rafaela Vasquez, wasn’t watching the road in the moments leading to the crash. She told NTSB investigators she was looking at the system's interface, which is built into the center console.
And so, along with the entire notion that robots can be safer drivers than humans, the crash casts doubt on a fundamental tenet of this nascent industry: that the best way to keep everyone safe in these early years is to have humans sitting in the driver’s seat, ready to leap into action.
This is fucking retarded.
What is the point of expecting a driver to just sit there with his hands on the wheel and be ready to take control over a vehicle at any given time when the car's programming just decides to crap out?
In further reading, the main problem is that the driverless vehicle operators are idiots who can't stop playing around with their phones while driving.
So they are put in a driverless car and are expected to sit through traffic and be expected to take control over the vehicle. Sounds like a really stupid game.
Dozens of companies are developing autonomous driving technology in the United States. They all rely on human safety drivers as backups. The odd thing about that reliance is that it belies one of the key reasons so many people are working on this technology. We are good drivers when we’re vigilant. But we’re terrible at being vigilant. We get distracted and tired. We drink and do drugs. We kill 40,000 people on US roads every year and more than a million worldwide. Self-driving cars are supposed to fix that. But if we can’t be trusted to watch the road when we’re actually driving, how did anyone think we’d be good at it when the robot’s doing nearly all the work?
“Of course this was gonna be a problem,” says Missy Cummings, the director of the Humans and Autonomy Laboratory and Duke Robotics at Duke University. “Your brain doesn’t like to sit idle. It is painful.”
In good faith, people really thought the safety drivers were going to do a good job.
In 2015, Cummings and fellow researchers ran their own test. “We put people in a really boring, four-lane-highway driving simulator for four hours, to see how fast people would mentally check out,” she says. On average, people dropped their guard after 20 minutes. In some cases, it took just eight minutes.
Everyone developing self-driving tech knows how bad humans are at focusing on the road. That’s why many automakers have declined to develop semiautonomous tech, where a car drives itself in a simple scenario like highway cruising, but needs a person to supervise and grab the wheel when trouble seems imminent. That kind of system conjures the handoff problem, and as Volvo’s head of safety and driver-assist technologies told WIRED in 2016, "That problem's just too difficult.”
The problem for the companies eager to skip that icky middle ground and go right for a fully driverless car is that they believe the only way to get there is by training on public roads—the testing ground that offers all the vagaries and oddities these machines must master. And the only reasonable approach—from a pragmatic and political point of view—to testing imperfect tech in two-ton vehicles speeding around other people is to have a human supervisor.
“I think, in good faith, people really thought the safety drivers were going to do a good job,” Cummings says. In a rush to move past the oh-so-fallible human, the people developing truly driverless cars doubled down on, yes, the oh-so-fallible human.
That’s why, before letting them on the road, Uber puts its vehicle operators through a three-week training course at its Pittsburgh R&D center. Trainees spend time in a classroom reviewing the technology and the testing protocols, and on the track learning to spot and avoid trouble. They even get a day at a racetrack, practicing emergency maneuvers at highway speeds. They’re taught to keep their hands an inch or two from the steering wheel, and the right foot over the brake. If they simply have to look at their phones, they’re supposed to take control of the car and put it in park first.
There’s a sense of complacency when you’re driving the same loops over and over, and you trust the vehicle.
Working alone in eight-hour shifts (in Phoenix they earn about $24 an hour), the babysitters are then set loose into the wild. Each day, they get a briefing from an engineer: Here’s where you’ll be driving, here’s what to look for. Maybe this version of the software is acting a bit funky around cyclists, or taking one particular turn a little fast.
And constantly, they are told: Watch the road. Don’t look at your phone. If you’re tired, stop driving. Uber also audits vehicle logs for traffic violations, and it has a full-time employee who does nothing but investigate potential infractions of the rules. Uber has fired drivers caught (by other operators or by people on the street) looking at their phones.
Still, the vigilance decrement proves persistent. “There’s fatigue, there’s boredom,” says one former operator, who left Uber recently and requested not to be named. “There’s a sense of complacency when you’re driving the same loops over and over, and you trust the vehicle.” That’s especially true now that Uber’s cars are, overall, pretty good drivers. This driver said that by early this year, the car would regularly go 20 miles without requiring intervention. If you’re tooling around the suburbs, that might mean an hour or more. As any RAF cadet watching a broken clock in a cabin could tell you, that’s a long time to stay focused. “You get lulled into a false sense of security,” the driver says.