Waymo Reveals the Technology Behind Its Robotaxi System

What’s lurking in the trunk of your car? Grocery bags? A spare tire? Jumper cables? How about a supercharged computer capable of executing up to one quadrillion operations per second? That’s the reality for Waymo.

In a recent reveal, Waymo shared intriguing insights into the powerful computing “brain” nestled in the trunks of its robotaxis, detailing chip architecture, processor specifications, and internal components. This information also highlights the hardware suppliers that contribute to the tech powering its fleet of autonomous vehicles.

Waymo stands at the forefront of driverless technology, currently operating around 4,000 vehicles across more than 10 cities and completing approximately 500,000 paid rides each week. The company asserts that its technology significantly lowers traffic accidents and injuries compared to human drivers. While much of the vehicle operation relies on extensive training and cloud-based simulations, a substantial amount of processing occurs directly within the vehicles.

Equipped with an array of cameras, lidar, and radar, Waymo’s robotaxis gather vast amounts of raw sensor data. This data is analyzed by artificial intelligence to quickly interpret the surroundings, transforming that information into safe driving commands in mere milliseconds. The computers installed in the trunks serve as the vehicle’s eyes, ears, and brain, providing a comprehensive 360-degree view and enabling rapid decision-making without human intervention.

“Compute systems for autonomous driving handle highly diverse workloads,” explain Satish Jeyachandran, VP of Engineering, and Daniel Rosenband, Compute Lead. “At Waymo, we are designing a state-of-the-art system that would be impressive even for a data center, with the added complexity of operating within a vehicle and meeting real-time demands.”

The design of Waymo’s compute system is guided by three core principles: responsiveness, ruggedness, and redundancy. The system must react to sensor data with minimal latency, necessitating immense computing power that Waymo has scaled by 20 times over the past eight years. It also needs to withstand rough road conditions and extreme temperatures, while running two systems in parallel to ensure reliability in case one fails. These principles are deemed “non-negotiable” by the executives.

Space is also a consideration; the system must not consume the entire trunk, as there are passengers and luggage to accommodate. Additionally, it needs to operate quietly to enhance the rider experience. While Waymo develops its compute stack in-house, it collaborates with various third-party suppliers for components it cannot manufacture itself.

Waymo’s explanation of its multi-sensor system’s real-time data fusion is particularly significant, especially in light of contrasting views from industry leaders. While some dismiss these systems as flawed, Waymo maintains that its robust and redundant architecture is what allows for large-scale deployment of its vehicles. “We built a machine learning-primary architecture to run advanced neural networks with minimal latency,” the executives state. “By pairing our ML technologies with top-tier CPUs, GPUs, and accelerators, we achieve a balanced, heterogeneous system.”

Waymo also introduced its custom silicon chip, a 5-nanometer ASIC designed to manage the massive influx of data from its sensors. Instead of sending raw data directly to the main processing unit, this chip cleans and combines sensor inputs while performing rapid AI checks as the information streams in. With a dedicated front-end processing capability of 1,000 TOPS (trillion operations per second), it integrates seamlessly into the overall system, ensuring quick reaction times.

Waymo’s executives assert that their system can swiftly process high-fidelity data from 13 high-resolution cameras “simultaneously and in real-time,” enhancing low-light perception and allowing the vehicle to detect more in the dark compared to traditional cameras.

In their conclusion, Jeyachandran and Rosenband list several hardware suppliers aiding in the development of Waymo’s advanced system: AMD, Micron, Nvidia, Samsung, SanDisk, Socionext, and TSMC. Rather than building its compute system entirely from scratch, Waymo focuses on real-time sensor fusion and front-end machine learning, leveraging the global supply chain for other components.

This glimpse into the technology powering Waymo’s fleet of robotaxis is fascinating, but one crucial detail remains unaddressed: the cost. Estimates suggest that the sixth-generation hardware could range from $20,000 to $25,000 per vehicle, a significant drop from the fifth generation’s estimated $100,000 to $125,000 per vehicle. However, these costs could still pose challenges for Waymo’s ongoing expansion.

Creating a brain to operate a fleet of driverless cars is an impressive feat, but turning that innovation into a profitable business presents a whole new set of challenges.