“We are being super paranoid about safety, so the date could shift,” Elon Musk tweeted on X, regarding Tesla’s June 22 robotaxi pilot in Austin that is tentatively planned. But under the veneer of this cautious prudence is a technological bet that might reshape not just Tesla’s future, but the very infrastructure of autonomous mobility.
Tesla’s strategy is nothing short of brazen. Whereas rivals such as Waymo and Zoox depend on a sensor-dense hardware platform cameras, radar, and lidar Tesla has bet on a vision-only, AI-first approach. As Musk proclaimed in a recent earnings call, “What will actually work best for the road system is artificial intelligence, digital neural nets, and cameras.” That doctrine, based on the assumption that human-level perception can be simulated by neural networks with huge real-world training data, forms the basis of the company’s Full Self-Driving (FSD) system.
Tesla’s vision system’s technical core is a convolutional neural network backbone supplemented by a proprietary “HydraNet” architecture. Contrary to modular sensor fusion pipelines preferred by competitors, Tesla’s network weaves together footage from eight high-resolution cameras to build a dynamic, three-dimensional representation of the car’s surroundings. The shared backbone allows the vehicle to execute sophisticated tasks lane detection, object recognition, semantic segmentation without the overt depth information supplied by lidar. The end product is a system that can, in theory, scale both quickly and cheaply. Tesla’s FSD v13 hardware stack, as of recent analysis, will cost only $400 per vehicle, while a lidar-based system would cost $9,300.
But this audacity comes with controversy. As the National Highway Traffic Safety Administration has so directly inquired, how will Tesla’s autonomous FSD differ from the supervised one, and what safety-related limitations geofencing, weather, time-of-day will be placed on it to prevent danger? The pilot will debut with only 10 to 20 Model Ys driving within a very geofenced area of Austin. Every car will be monitored remotely, with “plenty of tele-ops to ensure safety levels,” according to Morgan Stanley’s Adam Jonas following a recent Tesla investor conference. The teleoperation system, a human-in-the-loop safety net, is engineered to take over when the AI is faced with a situation that is uncertain or dangerous.
Teleoperation in itself is an engineering problem of some difficulty. Experience in the industry indicates that latency delays in the control signals are a key bottleneck. Missy Cummings, director of Mason’s Autonomy and Robotics Center, cautioned that “for teleoperation to be safe, the communication latency between the remote human driver and the car on the road has to be as low as 10 milliseconds,” a level that existing networks find challenging to meet. Tesla’s system is designed to combine selective remote driving with remote monitoring, using predictive algorithms like the Smith predictor in order to cover network delays and preserve controllability of the vehicle. In reality, this would mean that the car drives itself in normal circumstances, but a remote driver can take control or offer strategic input when the system is faced with an edge case a sudden construction site, say, or a complicated interaction with emergency responders.
The regulatory environment is still unclear. SAE has not explained yet if remote teleoperation is Level 2 (driver assist) or Level 4 (high degree of automation) autonomy. If the human in the remote location is in control, the system can be considered as Level 2, similar to Ford BlueCruise or GM Super Cruise. The vehicle needs to travel under specific conditions without the need for human intervention to qualify for Level 4 status. Tesla’s invite-only, geofenced Austin launch confined to public roads and a carefully selected user base is both technically conservative and regulatorily hedged.
The business model, on the other hand, is rather assertive. Musk is already imagining a hybrid fleet, mixing Tesla-owned robotaxis with owner-driver vehicles. “It’s a combination of a Tesla-owned fleet and also enabling Tesla owners to be able to add or subtract their car to the fleet,” Musk said in an interview with CNBC. The firm’s robotaxi app, previewed in a new video, enables users to call up a robotaxi and set temperature controls ahead of time. Car owners, Musk asserts, might earn more by putting their vehicle into the fleet than their lease fee, with some analysts estimating up to $40,000 per year in earnings per car under ideal utilization conditions if demand and payment rates are in sync. The model is similar to that of Uber and Airbnb, but with a twist: the “host” is an artificially intelligent car, and the “guest” will never know its owner.
But the road to scale is shrouded in doubt. Tesla’s internal safety data on FSD interventions are still unclear, and the company has not produced metrics similar to California’s required disengagement reports. Whereas the company asserts interventions are required solely once every 10,000 miles within the Austin pilot zone, anecdotal evidence from independent Tesla owners indicates a frequency more like once per 400 miles a discrepancy that prompts questions as to how the system will function outside the selective emphasis of the pilot.
The stakes are high. While General Motors’ Cruise sits on the sidelines and Waymo’s lidar-centric fleet delivers 250,000 trips a week in major U.S. cities, Tesla’s cost leadership and data-driven strategy represent a tantalizing, albeit untested, alternative. By being able to draw on over-the-air updates and a 2.5 million-strong connected car fleet, the company sets up a feedback loop for ongoing neural network training, which may advance the march toward true autonomy beyond what sensor-constrained competitors can hope to replicate.
As the initial Model Ys embark on their invite-only traversals of Austin’s roads, each mile will be under the microscope not only by investors and engineers but by regulators, safety activists, and the public. The result will not only depend on the acuity of Tesla’s neural nets but also on the durability of its teleoperation safety net and the company’s success in navigating the complicated dance of technology, regulation, and human trust.
