Researchers from Princeton University and the University of Washington have made a remarkable leap in camera technology by developing an ultra-compact camera—about the size of a grain of salt—that captures incredibly detailed, full-color images. Building on this breakthrough, the team has now developed a new type of camera designed for computer vision, a key area of artificial intelligence (AI) that enables computers to recognize and interpret images and videos.

This new prototype camera takes a radically different approach to computer vision. Unlike traditional devices that rely on electricity, this camera uses light to perform object identification and analysis, offering extraordinary speed and energy efficiency. The camera can identify objects at the speed of light, making it far faster and more efficient than conventional computer vision systems.

“The question for me was always how can we use algorithms to sense and understand the world,” explained Dr. Felix Heide, assistant professor of computer science at Princeton University and one of the authors of the study. “This is a completely new way of thinking about optics, which is very different from traditional optics,” added Dr. Arka Majumdar, professor of electrical and computer engineering and physics at the University of Washington, and the study’s second author. “It’s an end-to-end design, where the optics are designed in conjunction with the computational block.”

At the heart of this new camera system is the replacement of traditional camera lenses with specially engineered optics. The design incorporates 50 stacked, flat, and lightweight meta-lenses, each using microscopic nanostructures to manipulate light. These meta-lenses act as an optical neural network, enabling the camera to process data in a manner that mimics the human brain.

The key advantage of this new approach is that the meta-lenses allow much of the computation to take place within the optics themselves. By relying on light to do the processing, the system drastically reduces its reliance on traditional electricity-powered computational systems, making it both faster and more energy-efficient.

Heide’s interest in metasurfaces—artificial, sheet-like materials with sub-wavelength features—served as the inspiration for this breakthrough. Unlike traditional lenses made of glass or plastic, metasurfaces diffract light around tiny structures, much like light spreads when passing through a narrow slit. The unique geometry of these metasurfaces allows them to process light in novel ways, bypassing the need for large and energy-intensive lenses.

Working with experts from the Washington Nanofabrication Laboratory, the team engineered a metasurface lens that would replace traditional lenses with 50 stacked layers. The result is a system that not only processes light but does so with the efficiency of a neural network, able to identify and classify images at more than 200 times the speed of conventional methods.

What sets this technology apart is that the system doesn’t need to capture a perfect image in the way traditional cameras do. Instead of trying to record every detail, the metasurface lens acts as a filter, sorting optical data into categories such as edges, light and dark areas, and other features that may not even be perceptible to the human eye. This structured, pre-processed data is then used by the AI to classify objects with incredible accuracy and efficiency.

“We realized we don’t need to record a perfect image,” Heide explained. “We can record only certain features that we can then aggregate to perform tasks such as classification.”

The team’s new system uses less than 1% of the computing power typically required for conventional object recognition. The metasurface lens performs 99.4% of the workload, leaving only a minimal computational burden for the traditional AI system. The result is an AI-powered vision system that can perform hundreds of millions of calculations (FLOPS) instantly, far outpacing traditional models.

While conventional neural networks apply numerous mathematical filters (kernels) to extract useful data, requiring significant computational power even for a few pixels, this new system uses fewer, larger optical filters that naturally conduct complex image filtering as light passes through the lens. This allows the system to analyze entire images at once, without the need for constant recalculation.

Tiny pillars within each metasurface lens manipulate light without relying on electricity or active control, simplifying the design and making it vastly more energy-efficient. The integration of hardware and software is seamless, allowing for faster and more accurate image processing.

This breakthrough in light-based camera technology represents a new paradigm for computer vision. By integrating optics and computation in a way that was previously impossible, the system is poised to offer a host of benefits in applications ranging from autonomous vehicles to robotics, healthcare, and beyond. As AI continues to evolve, innovations like this camera could significantly reduce the computational demands of real-time image analysis while enhancing speed and energy efficiency.

By Impact Lab