Open Vocabulary Robots Hit New Benchmark

A humanoid now recognizes unseen objects in real time without a predefined label set.

Testing shows a practical path toward open world perception, according to a new prompt free system called AnomNOVIC. The two stage approach blends a masked autoencoder that generates generic, object agnostic bounding boxes with NOVIC, a real time open vocabulary image classifier that operates without a predefined candidate class list. In a tabletop setup using the NICOL humanoid, the researchers report 47.1% AP and 57.5% AP50 for prompt free recognition. When a list of candidate classes is provided, performance climbs to 59.0% AP and 72.5% AP50, highlighting a familiar engineering tradeoff between openness and accuracy. In a separate in the wild test with 48 unique objects, AnomNOVIC reaches up to 82.6% prompt free detection and classification accuracy, a noteworthy improvement over established baselines such as YOLO World v2, OWLv2 and YOLOE.

The system is described as two stage by design. It relies on a masked autoencoder to produce bounding boxes that are not tied to any known set of objects, creating a flexible region proposal mechanism. NOVIC then takes those regions and attempts to classify what is salient in each, without being tethered to a fixed catalog of labels. The result is a recognition pipeline that can respond to objects the robot has never seen before, provided the scene yields informative visual cues. The NICOL humanoid serves as the testbed, a controlled environment that mirrors practical tabletop manipulation tasks and object interactions rather than a static image benchmark.

From an engineering viewpoint, the key shift is moving away from curated class lists toward genuine open vocabulary perception that still lands concrete, actionable labels. The data show that having a finite candidate set boosts accuracy, but the core capability remains intact without it. That matters for real world deployments where a robot encounters novel items, tools, or packaging in factories, warehouses, or homes. The paper notes that the prompt free mode already outperforms competitive baselines on multiple datasets, indicating the approach scales beyond a single lab scenario.

For practitioners, several concrete takeaways emerge. First, the two stage design delivers a measurable gain in feasibility: a generic box proposer paired with a flexible classifier can yield useful identifications without expensive re annotation. Second, opt in to a restricted candidate list when available to squeeze higher accuracy, but design autonomous systems that can still function meaningfully when such guidance is absent. Third, the open world promise hinges on robust bounding boxes; errors at the proposal stage propagate, so MAE quality and the criteria used for box generation will be a focal point in deployment. Fourth, compute and latency matter; a prompt free pipeline needs to remain real time on the robot’s hardware, not just in a lab compute cluster.

Looking ahead, observers say AnomNOVIC represents a substantial engineering milestone toward open world autonomy for humanoids. The immediate next steps will test the approach across additional hardware platforms and in more varied environments, including cluttered scenes and longer duration sessions, to evaluate resilience to occlusions, lighting changes, and object wear. If the gains hold, expect further integration with manipulation stacks so a robot not only detects unknown objects but decides how to interact with them in real time, a key plank for practical, scalable humanoid action.

The result is a concrete demonstration that open vocabulary perception can be made practical for real robots, not just an academic curiosity. The method shows what changes in practice when you replace fixed label taxonomies with a robust, real time recognition flow that can adapt on the fly.