The research model forecasts future 3D scene structure while generating actions, then uses that forecast as training guidance.
A new paper on arXiv presents E0.5, a proposed research model for robot control—not a product available for purchase. Its central idea is to make a 3D foundation model the system’s main “map” of the scene, rather than adding geometry to a video or language model later.
E0.5 predicts a future scene state alongside each robot action sequence. That state includes depth, 3D point maps, and camera information at the end of an action chunk. In plain terms, the model learns not only what the robot sees now, but how the scene should look after the robot moves.
The future scene is a training target, not an extra sensor feed required during operation. The authors state that “future geometry serves exclusively as training supervision and is not required during inference.” This keeps the deployed control loop dependent on current observations, instructions, and robot state.
On the simulated RoboTwin 2.0 bimanual-manipulation benchmark, the authors report success rates of 89.90% in clean scenes and 50.09% in randomized scenes. Their ablation study reports that future-geometry prediction raised average success by 9.43 percentage points over removing geometric reconstruction, and by 7.93 points over reconstructing only the current scene.
The paper also reports physical-robot experiments on AgileX Piper dual-arm systems and Zerith H1 humanoid robots. Pretraining on 160,010 trajectories across 177 tasks reportedly raised mean real-world success by about 15 percentage points on Piper and seven on H1.
These are author-reported experiments, not independent validation or production deployment. The large pretraining study uses model-generated depth estimates as geometric supervision, so broader testing with measured geometry remains an important next step.