Wayve, a leading innovator in Embodied AI for self-driving technologies, has announced the launch of PRISM-1, a groundbreaking 4D reconstruction model designed to significantly enhance the testing and training of its Advanced Driver Assistance Systems (ADAS) and autonomous driving technology. PRISM-1 represents a major advancement in 4D reconstruction, enabling scalable and realistic resimulations of complex driving scenes with minimal engineering or labeling input.

First showcased in December 2023 through Wayve’s Ghost Gym neural simulator, PRISM-1 employs novel view synthesis to create precise 4D scene reconstructions (3D in space plus time) using only camera inputs. This method promises to revolutionize simulation for autonomous driving by accurately and efficiently simulating the dynamics of complex and unstructured real-world environments. PRISM-1 powers the next generation of Ghost Gym simulations and departs from traditional methods that rely on LiDAR and 3D bounding boxes. Instead, it uses novel view synthesis techniques to accurately depict moving elements such as pedestrians, cyclists, vehicles, and traffic lights, capturing precise details like clothing patterns, brake lights, and windshield wipers.

Achieving high realism is crucial for building an effective training simulator and evaluating driving technologies. Traditional simulation technologies often treat vehicles as rigid entities, failing to capture critical dynamic behaviors like indicator lights or sudden braking. PRISM-1, however, uses a flexible framework that excels at identifying and tracking changes in the appearance of scene elements over time. This enables it to resimulate complex dynamic scenarios with elements that change in shape and move throughout the scene. PRISM-1 distinguishes between static and dynamic elements in a self-supervised manner, avoiding the need for explicit labels, scene graphs, and bounding boxes to define the configuration of a busy street. This approach maintains efficiency even as scene complexity increases, ensuring that more complex scenarios do not require additional engineering effort, making PRISM-1 a scalable and efficient solution for simulating complex urban environments.

Jamie Shotton, Chief Scientist at Wayve, commented on the significance of PRISM-1: “PRISM-1 bridges the gap between the real world and our simulator. By enhancing our simulation platform with accurate dynamic representations, Wayve can extensively test, validate, and fine-tune our AI models at scale. We are building Embodied AI technology that generalizes and scales. To achieve this, we continue to advance our end-to-end AI capabilities, not only in our driving models but also through enabling technologies like PRISM-1.”

In conjunction with PRISM-1’s launch, Wayve is also releasing its WayveScenes101 Benchmark, a dataset comprising 101 diverse driving scenarios from the UK and US. These scenarios include urban, suburban, and highway scenes under various weather and lighting conditions. Wayve aims for this dataset to support the AI research community in advancing novel view synthesis models and developing more robust and accurate scene representation models for driving.

PRISM-1 and the WayveScenes101 dataset mark significant milestones in the field of autonomous driving, offering powerful tools to foster innovation and enhance the development of safer and more efficient self-driving technologies.

By Impact Lab