NVIDIA Research introduces Neuralangelo, their latest AI breakthrough that can transform ordinary 2D iPhone videos into intricate and lifelike 3D structures, replicating real-world objects and environments with astonishing accuracy.
Inspired by Michelangelo’s ability to sculpt masterpieces from marble blocks, Neuralangelo employs advanced algorithms to generate complex 3D structures and textures from simple video clips. These realistic 3D objects and spaces can then be seamlessly integrated into various design applications, enabling the creation of art, video games, robotics, and digital twins.
Building upon NVIDIA Research’s previous projects, such as the NVIDIA 3D MoMa tool for converting photos into 3D objects, Neuralangelo furthers the concept by importing more detailed spaces and objects. What sets Neuralangelo apart is its remarkable capability to capture repeating texture patterns, consistent colors, and significant color variations, areas where previous AI models have faced challenges.
Neuralangelo leverages the technology behind NVIDIA Instant NeRF, an instant neural graphics primitive, to capture these intricate details. The tool can even convert drone footage of vast locations into massive 3D objects.
Here’s how it works: Neuralangelo analyzes video footage of an object or scene from different angles. It selects multiple frames with varying perspectives, determines the camera positions in each frame, and constructs a preliminary 3D representation of the scene. The model then optimizes the rendering to enhance details and generates the final high-fidelity 3D object.
NVIDIA highlights that Neuralangelo’s ability to accurately translate complex material textures, including roof shingles, glass panes, and smooth marble, from 2D video to 3D assets surpasses previous methods.
The high fidelity and ease of use make Neuralangelo a valuable tool for developers and creative professionals, enabling them to rapidly create detailed 3D reconstructions using smartphone footage.
It revolutionizes generating virtual objects for various projects, offering various applications across different industries.
