Summary
NVIDIA has presented developer projects using AI agents with Omniverse libraries to assemble and refine simulation applications. The examples range from humanoid-robot and autonomous-driving tests to digital twins, with NVIDIA staff directing and reviewing the work.
NVIDIA has showcased developer projects in which AI agents help assemble simulation applications using NVIDIA Omniverse libraries. In examples published on October 8, 2026, agents generated application code, connected simulation components and revised digital scenes in response to tests. NVIDIA staff directed the work, checked integrations and guided changes.
The projects span humanoid robots, autonomous driving, sensor comparisons, product disassembly and other uses. They illustrate how agents can help connect the parts of a simulation workflow: assets, physics, rendering, sensors and user interfaces.
How developers used the agents
For a warehouse humanoid simulator, NVIDIA Omniverse product manager Frank DeLise used GPT-6 Astra to connect libraries for physics (ovphysx), scene updates (ovstage), rendering (ovrtx) and the user interface (ovui). Astra also generated animation and application code for a scene using a SimReady warehouse and robot. The resulting prototype offered first- and third-person views for exploring warehouse tasks.
In an autonomous-driving example, NVIDIA simulation technology manager Doyub Kim asked Astra to build a reusable test environment based on San Francisco’s Market Street. He directed it to connect asset creation, traffic, RTX sensor simulation and the Alpamayo driving model in stages, checking each integration. NVIDIA describes the result as a testbed for comparing models and examining how changes to a scene or sensors affect driving behavior. In a separate experiment, Cosmos3-Nano varied weather and lighting in recorded simulation videos so the driving model’s responses could be compared under different conditions.
Another project brought NASA assets into an OpenUSD model of the International Space Station, with telemetry presented in a browser application. NVIDIA engineering director Nic Johns used prompts and follow-up corrections to guide assembly of the scene and its interface. The workflow used Blender for asset preparation and Omniverse libraries for rendering, scene runtime and streaming.
Using simulation results to refine designs
Several examples use measurements or simulated trials to guide revisions. NVIDIA’s Ashley Reid directed Astra and Claude Fable 5 agents to compare camera and raw LiDAR outputs from ovrtx with recorded data. Over about three days, the agents created two digital twins and improved two existing ones, making changes to missing objects, geometry and materials. Acceptance depended on camera and LiDAR metrics. In this context, a digital twin is a digital scene built to represent a real environment closely enough for comparisons and testing.
For its experimental Robo Olympics project, NVIDIA used sports videos and natural-language instructions to guide Astra in building simulated Unitree G1 humanoids performing sports movements. Astra built and refined controllers through physics trials. In one experiment, a robot cleared a single hurdle in 64 of 100 simulation trials; NVIDIA says the trials provided feedback for adjusting timing and control.
NVIDIA also describes using Astra to model a car suspension in PTC Onshape and configure it in Isaac Sim. The agent measured available space and designed a wrench for reaching the suspension bolts; NVIDIA’s Jens Jebens reported successful removal of a component in simulation. In a separate room-reconstruction workflow, captured geometry was turned into an editable scene, with Isaac Sim tests guiding revisions to door and drawer collisions and contact behavior.
Across the examples, the developer remained part of the loop: people supplied goals, checked results and steered revisions. The significance is practical rather than a demonstration of fully automated engineering: agents were used to connect tools and produce editable test environments, while simulation results gave developers material to inspect and improve.