# BINDER: Instantly Adaptive Mobile Manipulation with Open-Vocabulary Commands

arXiv: https://arxiv.org/abs/2511.22364 · HTML: https://mlsanity.com/paper/2511.22364/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 39 references that are in the map.
- First posted on arXiv 27 Nov 2025.

## Builds on (most cited first, 29 of 29)

- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [π0: A Vision-Language-Action Flow Model for General Robot Control](https://mlsanity.com/paper/2410.24164/) (2024)
- [Code as Policies: Language Model Programs for Embodied Control](https://mlsanity.com/paper/2209.07753/) (2022)
- [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020)
- [ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks](https://mlsanity.com/paper/1912.01734/) (2019)
- [Visual Language Maps for Robot Navigation](https://mlsanity.com/paper/2210.05714/) (2022)
- [ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning](https://mlsanity.com/paper/2309.16650/) (2023)
- [TidyBot: Personalized Robot Assistance with Large Language Models](https://mlsanity.com/paper/2305.05658/) (2023)
- [Hi Robot: Open-Ended Instruction Following with Hierarchical Vision-Language-Action Models](https://mlsanity.com/paper/2502.19417/) (2025)
- [AnyGrasp: Robust and Efficient Grasp Perception in Spatial and Temporal Domains](https://mlsanity.com/paper/2212.08333/) (2022)
- [Hierarchical Open-Vocabulary 3D Scene Graphs for Language-Grounded Robot Navigation](https://mlsanity.com/paper/2403.17846/) (2024)
- [CLIP-Fields: Weakly Supervised Semantic Fields for Robotic Memory](https://mlsanity.com/paper/2210.05663/) (2022)
- [Open-vocabulary Queryable Scene Representations for Real World Planning](https://mlsanity.com/paper/2209.09874/) (2022)
- [HomeRobot: Open-Vocabulary Mobile Manipulation](https://mlsanity.com/paper/2306.11565/) (2023)
- [How NeRFs and 3D Gaussian Splatting are Reshaping SLAM: a Survey](https://mlsanity.com/paper/2402.13255/) (2024)
- [Language-Grounded Dynamic Scene Graphs for Interactive Object Search with Mobile Manipulation](https://mlsanity.com/paper/2403.08605/) (2024)
- [Back-tracing Representative Points for Voting-based 3D Object Detection in Point Clouds](https://mlsanity.com/paper/2104.06114/) (2021)
- [Code-as-Monitor: Constraint-aware Visual Programming for Reactive and Proactive Robotic Failure Detection](https://mlsanity.com/paper/2412.04455/) (2024)
- [Open-Vocabulary Functional 3D Scene Graphs for Real-World Indoor Spaces](https://mlsanity.com/paper/2503.19199/) (2025)
- [Closed-Loop Visuomotor Control with Generative Expectation for Robotic Manipulation](https://mlsanity.com/paper/2409.09016/) (2024)
- [RACER: Rich Language-Guided Failure Recovery Policies for Imitation Learning](https://mlsanity.com/paper/2409.14674/) (2024)
- [Closed-Loop Open-Vocabulary Mobile Manipulation with GPT-4V](https://mlsanity.com/paper/2404.10220/) (2024)
- [Dynamic Open-Vocabulary 3D Scene Graphs for Long-term Language-Guided Mobile Manipulation](https://mlsanity.com/paper/2410.11989/) (2024)
- [DynaMem: Online Dynamic Spatio-Semantic Memory for Open World Mobile Manipulation](https://mlsanity.com/paper/2411.04999/) (2024)
- [Open-vocabulary Mobile Manipulation in Unseen Dynamic Environments with 3D Semantic Maps](https://mlsanity.com/paper/2406.18115/) (2024)
- [MORE: Mobile Manipulation Rearrangement Through Grounded Language Reasoning](https://mlsanity.com/paper/2505.03035/) (2025)
- [From Grounding to Manipulation: Case Studies of Foundation Model Integration in Embodied Robotic Systems](https://mlsanity.com/paper/2505.15685/) (2025)

## Abstract (arXiv)

Open-vocabulary mobile manipulation (OVMM) requires robots to follow language instructions, navigate, and manipulate while updating their world representation under dynamic environmental changes. However, most prior approaches update their world representation only at discrete update points such as navigation targets, waypoints, or the end of an action step, leaving robots blind between updates and causing cascading failures: overlooked objects, late error detection, and delayed replanning. To address this limitation, we propose BINDER (Bridging INstant and DEliberative Reasoning), a dual process framework that decouples strategic planning from continuous environment monitoring. Specifically, BINDER integrates a Deliberative Response Module (DRM, a multimodal LLM for task planning) with an Instant Response Module (IRM, a VideoLLM for continuous monitoring). The two modules play complementary roles: the DRM performs strategic planning with structured 3D scene updates and guides what the IRM attends to, while the IRM analyzes video streams to update memory, correct ongoing actions, and trigger replanning when necessary. Through this bidirectional coordination, the modules address the trade off between maintaining awareness and avoiding costly updates, enabling robust adaptation under dynamic conditions. Evaluated in three real world environments with dynamic object placement, BINDER achieves substantially higher success and efficiency than SoTA baselines, demonstrating its effectiveness for real world deployment.