# UniTeam: Open Vocabulary Mobile Manipulation Challenge

arXiv: https://arxiv.org/abs/2312.08611 · HTML: https://mlsanity.com/paper/2312.08611/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 17 references that are in the map.
- First posted on arXiv 14 Dec 2023.

## Builds on (most cited first, 14 of 14)

- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [Detecting Twenty-thousand Classes using Image-level Supervision](https://mlsanity.com/paper/2201.02605/) (2022)
- [ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning](https://mlsanity.com/paper/2309.16650/) (2023)
- [Recognize Anything: A Strong Image Tagging Model](https://mlsanity.com/paper/2306.03514/) (2023)
- [HomeRobot: Open-Vocabulary Mobile Manipulation](https://mlsanity.com/paper/2306.11565/) (2023)
- [The Design of Stretch: A Compact, Lightweight Mobile Manipulator for Indoor Human Environments](https://mlsanity.com/paper/2109.10892/) (2021)
- [Contrastive Language, Action, and State Pre-training for Robot Learning](https://mlsanity.com/paper/2304.10782/) (2023)
- [Critic Guided Segmentation of Rewarding Objects in First-Person Views](https://mlsanity.com/paper/2107.09540/) (2021)
- [Solving Physics Puzzles by Reasoning about Paths](https://mlsanity.com/paper/2011.07357/) (2020)
- [Behavioral Cloning via Search in Video PreTraining Latent Space](https://mlsanity.com/paper/2212.13326/) (2022)
- [Planning with RL and episodic-memory behavioral priors](https://mlsanity.com/paper/2207.01845/) (2022)
- [Modularization of End-to-End Learning: Case Study in Arcade Games](https://mlsanity.com/paper/1901.09895/) (2019)
- [Language-Conditioned Semantic Search-Based Policy for Robotic Manipulation Tasks](https://mlsanity.com/paper/2312.05925/) (2023)
- [Behavioral Cloning via Search in Embedded Demonstration Dataset](https://mlsanity.com/paper/2306.09082/) (2023)

## Built on by (most cited first, 15 of 15)

- [OK-Robot: What Really Matters in Integrating Open-Knowledge Models for Robotics](https://mlsanity.com/paper/2401.12202/) (2024)
- [DynaMem: Online Dynamic Spatio-Semantic Memory for Open World Mobile Manipulation](https://mlsanity.com/paper/2411.04999/) (2024)
- [3D CAVLA: Leveraging Depth and 3D Context to Generalize Vision Language Action Models for Unseen Tasks](https://mlsanity.com/paper/2505.05800/) (2025)
- [Towards Open-World Mobile Manipulation in Homes: Lessons from the Neurips 2023 HomeRobot Open Vocabulary Mobile Manipulation Challenge](https://mlsanity.com/paper/2407.06939/) (2024)
- [ConceptAgent: LLM-Driven Precondition Grounding and Tree Search for Robust Task Planning and Execution](https://mlsanity.com/paper/2410.06108/) (2024)
- [HiFi-CS: Towards Open Vocabulary Visual Grounding For Robotic Grasping Using Vision-Language Models](https://mlsanity.com/paper/2409.10419/) (2024)
- [Digital Twin Generation from Visual Data: A Survey](https://mlsanity.com/paper/2504.13159/) (2025)
- [Natural Language as Policies: Reasoning for Coordinate-Level Embodied Control with LLMs](https://mlsanity.com/paper/2403.13801/) (2024)
- [Cognitive Planning for Object Goal Navigation using Generative AI Models](https://mlsanity.com/paper/2404.00318/) (2024)
- [DM2RM: Dual-Mode Multimodal Ranking for Target Objects and Receptacles Based on Open-Vocabulary Instructions](https://mlsanity.com/paper/2408.07910/) (2024)
- [OWMM-Agent: Open World Mobile Manipulation With Multi-modal Agentic Data Synthesis](https://mlsanity.com/paper/2506.04217/) (2025)
- [SplatR : Experience Goal Visual Rearrangement with 3D Gaussian Splatting and Dense Feature Matching](https://mlsanity.com/paper/2411.14322/) (2024)
- [Grounding Language Models with Semantic Digital Twins for Robotic Planning](https://mlsanity.com/paper/2506.16493/) (2025)
- [Object and Contact Point Tracking in Demonstrations Using 3D Gaussian Splatting](https://mlsanity.com/paper/2411.03555/) (2024)
- [EgoActor: Grounding Task Planning into Spatial-aware Egocentric Actions for Humanoid Robots via Visual-Language Models](https://mlsanity.com/paper/2602.04515/) (2026)

## Abstract (arXiv)

This report introduces our UniTeam agent - an improved baseline for the "HomeRobot: Open Vocabulary Mobile Manipulation" challenge. The challenge poses problems of navigation in unfamiliar environments, manipulation of novel objects, and recognition of open-vocabulary object classes. This challenge aims to facilitate cross-cutting research in embodied AI using recent advances in machine learning, computer vision, natural language, and robotics. In this work, we conducted an exhaustive evaluation of the provided baseline agent; identified deficiencies in perception, navigation, and manipulation skills; and improved the baseline agent's performance. Notably, enhancements were made in perception - minimizing misclassifications; navigation - preventing infinite loop commitments; picking - addressing failures due to changing object visibility; and placing - ensuring accurate positioning for successful object placement.