NVIDIA has introduced a groundbreaking AI model called Neuralangelo, capable of generating 3D replicas of objects from 2D videos. Whether the subject is a classical sculpture or a regular truck or building, Neuralangelo utilizes multiple frames from different angles to capture the depth, size, and shape of the object. It then creates an initial 3D representation, refining it to mimic the intricate details of the real-world counterpart.

By leveraging technology from its previous model, Instant NeRF, Neuralangelo achieves remarkable precision in capturing fine details, such as textures, patterns, and color variations. NVIDIA claims that Neuralangelo’s ability to handle challenging textures, like the roughness of roof shingles or the smoothness of marble, “significantly surpasses prior methods.” The model can even generate usable virtual objects from smartphone videos, providing creators with a simple way to produce 3D replicas for their projects. Moreover, NVIDIA Research states that Neuralangelo can generate complete 3D scenes within a remarkably short timeframe of two hours.

Ming-Yu Liu, co-author of the Neuralangelo research paper, envisions that the model will eventually enable developers to import highly detailed objects, ranging from small statues to massive buildings, into virtual environments for video games or industrial digital twins. NVIDIA’s demonstration video showcases the ability to create expansive landscapes using drone footage, facilitating the recreation of real-world locations in game development. The potential applications of Neuralangelo extend beyond gaming and include robotics, virtual reality experiences, architecture, and various artistic endeavors.

NVIDIA’s research arm plans to present Neuralangelo at the Conference on Computer Vision and Pattern Recognition event held from June 18th to the 22nd in Vancouver. This unveiling marks a significant advancement in the field of AI-generated 3D replication, with Neuralangelo offering exciting possibilities for developers and creators across diverse industries.

By Impact Lab