Remember the advanced, intelligent and almost indestructible car in the 1980s Knight Rider TV drama? KITT (Knight Industries Two Thousand) was a car way ahead of its time, powered by artificial intelligence and loaded with advanced features and weaponry.
Fast forward to 2021 and NVIDIA Research has managed to resurrect the icon — by using a new deep learning engine that creates 3D object models from standard 2D images in NVIDIA Omniverse.
The GANverse3D application inflates flat images into realistic 3D models that can be visualised and controlled in virtual environments. A single photo of a car is all that’s needed to generate a 3D model that can drive around in virtual world.
A generative adversarial network (GAN) is harnessed to synthesise images depicting the same object from multiple viewpoints. These images are then plugged into a rendering framework for inverse graphics, the process of inferring 3D mesh models from 2D images. Once trained on multi-view images, GANverse3D needs only a single 2D image to predict a 3D mesh model.
When imported as an extension in NVIDIA Omniverse and run on NVIDIA RTX GPUs, GANverse3D can be used to recreate any 2D image into 3D.
“We turned a GAN model into a very efficient data generator so we can create 3D objects from any 2D image on the web,” said Wenzheng Chen, Research Scientist of NVIDIA and Lead Author on the project.
Architects, creators, game developers and designers can easily add new objects to their mockups without needing expertise in 3D modeling, or a large budget to spend on renderings. They can use virtual environments such as NVIDIA Omniverse to test out new ideas and visualise prototypes before creating their final products. And with Omniverse Connectors, developers can use their preferred 3D applications in Omniverse to simulate complex virtual worlds with real-time ray tracing.
Entelechy Asia 