Virtual data trains a humanoid to climb stairs

A humanoid trained entirely in virtual data just climbed stairs with 90 percent real-world success.

The achievement comes from GRAIL, a digital generation pipeline designed to stay in simulation until deployment. By composing 3D assets, simulator-ready scenes, and priors from video foundation models, GRAIL can synthesize human-robot interactions without rebuilding physical environments or teleoperating a robot. A core idea is a privileged, fully specified starting point: object geometry, camera parameters, metric scale, environment depth, and a robot-proportioned character are known before video generation and re-used during 4D reconstruction. That setup helps the system recover metric, time-aligned trajectories of human-object interactions with less depth ambiguity and fewer morphology mismatches. The recovered motions are then retargeted to a humanoid robot and paired with two complementary trackers: a latent object-aware adaptor for manipulation and a scene-aware tracker for terrain traversal. In total, GRAIL produced more than 20,000 synthetic sequences covering pick-up, manipulation, sitting, and terrain traversal. In a sim-to-real pipeline, researchers trained egocentric visual policies entirely in simulation and deployed them on a Unitree G1 humanoid, achieving 84 percent real-world success on diverse object pick-up and 90 percent on stair climbing. Testing shows the approach can turn virtual data into practical motion on real hardware.

From an engineering standpoint, the punchline is a shift in where data comes from and how the system is trained. GRAIL avoids the bottleneck of teleoperation campaigns and instrumented actors by generating scenes that are consistent, measurable, and repeatable. The virtual-first approach also constrains the kinds of tasks that can be learned from day one: it relies on precise 3D configurations and scene depth available during generation, which reduces depth ambiguity during recovery but may limit immediate generalization to very different environments. The team retargets motions to the Unitree platform and tunes two trackers to stay aware of objects and terrain, a design choice that helps keep the policy robust when the robot encounters unfamiliar obstacles in the real world.

Practitioner insights to watch as the approach matures include several concrete tradeoffs. First, the sim-to-real bridge relies on high-fidelity 3D configurations and accurate depth priors; any mismatch between generated scenes and real environments could erode performance on tasks beyond those represented in the synthetic set. Second, generating and curating 20,000 sequences demands substantial compute and asset management, even before any real-robot testing begins, which factors into project cost and cadence. Third, while the Unitree G1 serves as a valuable testbed, translating the method to other platforms will stress differences in actuator dynamics, balance control, and payload constraints, elements not detailed in the paper but critical in production deployments. Finally, the approach heavy-lifts perception by depending on video foundation models to infer interactions; improvement in perception reliability will be essential as tasks add clutter, occlusion, and dynamic lighting.

If GRAIL scales, the next watchpoints are broader scene diversity, heavier manipulation under real-time constraints, and longer-horizon tasks that require planning across rooms or multi-step interactions. The core promise remains: a firmly engineering path from synthetic data to real-world capability, with explicit metrics and a pathway to repeatable gains in task-general locomotion and loco-manipulation for humanoids.