RoboSimStudio
An open interactive toolkit that accelerates agentic physics-based Real2Sim of Robot Manipulation Tasks
1University of California, Berkeley · 2NVIDIA · *Equal contribution
1University of California, Berkeley · 2NVIDIA · *Equal contribution
RoboSimStudio takes a smartphone video of a robot performing a task, here wiping a bar with a sponge, and a one-line prompt to a coding agent.
A Claude or Codex agent builds the simulation from the video: the scene, the camera and the physics, in about an hour (Real2Sim).
The output is a multi-physics simulation: rigid bodies in Newton coupled with the FEM sponge in SuperDex, which humans and agents inspect and tune until it matches the video.
See a rollout of the simulation. Use A W S D and drag to look around!
Varied in its physical parameters and layout, the simulation improves a Graph-as-Policy from feedback, and the policy runs on the real robot (Sim2Real).
Loading…