Robot World Models Need Friction Inputs, Not Just Better Vision

VμA proposes making friction a first-class conditioning input for robot world models handling contact-rich tasks.

Robot world models may need to account for friction more directly if they are to handle unfamiliar objects and surfaces reliably.

The proposed VμA model class treats friction as a conditioning signal for robot learning systems. Its premise is that a robot cannot reliably predict the result of a grasp, slide, push, or placement without information about the physical interaction at the contact surface.

Current robot world-model conditioning relies primarily on camera observations and end-effector position from joint encoders, according to the proposal. Those inputs can be sufficient for free-space motion, where a robot is moving without touching objects.

Contact-rich manipulation presents a different problem. Once a robot touches an object, the information that matters includes the conditions at the interface between the robot and the object. Visual observations and joint positions alone may not fully describe those conditions.

Friction can determine whether contact remains stable or becomes a slip. A robot may encounter surfaces or objects that appear similar visually but behave differently during manipulation. Making friction an explicit input could help a model better predict the consequences of its actions in those situations.

The broader goal of world models is to give robots an internal representation of physical reality that can support prediction, planning, and generalization beyond previously seen tasks. But those predictions depend on how well the model is conditioned on the robot’s current physical state.

VμA focuses on a gap in that conditioning: touch without friction information may remain incomplete for reliable manipulation across novel surfaces and objects.

The proposal does not, in the available description, provide technical details about VμA’s architecture, how friction is measured or estimated, or benchmark results demonstrating improved manipulation performance. Its central argument, however, is clear: for robots working in contact with the world, friction may be an essential part of the state they need to understand.