Data to 7 Oct 2026

BUMBLE: Unifying Reasoning and Acting with Vision-Language Models for Building-wide Mobile Manipulation

arXiv 2410.06237 ↗First posted 8 Oct 2024cs.RO cs.AIcited by 31

On the map: LLM reasoning → Spatial and embodied reasoning

also in 1 more

Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

To operate at a building scale, service robots must perform very long-horizon mobile manipulation tasks by navigating to different rooms, accessing different floors, and interacting with a wide and unseen range of everyday objects. We refer to these tasks as Building-wide Mobile Manipulation. To tackle these inherently long-horizon tasks, we introduce BUMBLE, a unified Vision-Language Model (VLM)-based framework integrating open-world RGBD perception, a wide spectrum of gross-to-fine motor skills, and dual-layered memory. Our extensive evaluation (90+ hours) indicates that BUMBLE outperforms multiple baselines in long-horizon building-wide tasks that require sequencing up to 12 ground truth skills spanning 15 minutes per trial. BUMBLE achieves 47.1% success rate averaged over 70 trials in different buildings, tasks, and scene layouts from different starting rooms and floors. Our user study demonstrates 22% higher satisfaction with our method than state-of-the-art mobile manipulation methods. Finally, we demonstrate the potential of using increasingly-capable foundation models to push performance further. For more information, see https://robin-lab.cs.utexas.edu/BUMBLE/

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 29 of its 78 references that are in the map. First posted on arXiv 8 Oct 2024.

Builds on

29 of 78 references.

Closest ancestors · 3

Same direction, newest first

Other references · 26

2023Segment Anythingcited by 8,158
2021On the Opportunities and Risks of Foundation Modelscited by 3,961
2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,151
2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
2019Dynamics-Aware Unsupervised Discovery of Skillscited by 308
2023Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4Vcited by 291
2023SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planningcited by 289
2023Look Before You Leap: Unveiling the Power of GPT-4V in Robotic Vision-Language Planningcited by 161
2023Open-World Object Manipulation using Pre-trained Vision-Language Modelscited by 155
2024MOKA: Open-World Robotic Manipulation through Mark-Based Visual Promptingcited by 126

14 more

2024CoPa: General Robotic Manipulation through Spatial Constraints of Parts with Foundation Modelscited by 97
2017Neural Task Programming: Learning to Generalize Across Hierarchical Taskscited by 75
2021Demonstration-Guided Reinforcement Learning with Learned Skillscited by 73
2024Language-Grounded Dynamic Scene Graphs for Interactive Object Search with Mobile Manipulationcited by 69
2024Learning From Failure: Integrating Negative Examples when Fine-tuning Large Language Models as Agentscited by 49
2024Vision-based Manipulation from Single Human Video with Open-World Object Graphscited by 46
2024Closed-Loop Open-Vocabulary Mobile Manipulation with GPT-4Vcited by 40
2024OK-Robot: What Really Matters in Integrating Open-Knowledge Models for Roboticscited by 33
2019Regression Planning Networkscited by 32
2024TeleMoMa: A Modular and Versatile Teleoperation System for Mobile Manipulationcited by 27
2024Open-vocabulary Mobile Manipulation in Unseen Dynamic Environments with 3D Semantic Mapscited by 13
2024Demonstrating Mobile Manipulation in the Wild: A Metrics-Driven Approachcited by 10
2024MOSAIC: Modular Foundation Models for Assistive and Interactive Cookingcited by 9
2023PDSketch: Integrated Planning Domain Programming and Learningcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 19; 2026: 11.

31 papers.

Sort:

By area:

2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
2025Hi Robot: Open-Ended Instruction Following with Hierarchical Vision-Language-Action Modelscited by 216
2024GraphEQA: Using 3D Semantic Scene Graphs for Real-time Embodied Question Answeringcited by 37
2025VLA-OS: Structuring and Dissecting Planning Representations and Paradigms in Vision-Language-Action Modelscited by 34
2025RoboCerebra: A Large-scale Benchmark for Long-horizon Robotic Manipulation Evaluationcited by 31
2025BEHAVIOR Robot Suite: Streamlining Real-World Whole-Body Manipulation for Everyday Household Activitiescited by 29
2025Robix: A Unified Model for Robot Interaction, Reasoning and Planningcited by 23
2025Towards Embodied Agentic AI: Review and Classification of LLM- and VLM-Driven Robot Autonomy and Interactioncited by 18
2025MORE: Mobile Manipulation Rearrangement Through Grounded Language Reasoningcited by 9
2026MANSION: Multi-floor lANguage-to-3D Scene generatIOn for loNg-horizon taskscited by 8
2025Subtask-Aware Visual Reward Learning from Segmented Demonstrationscited by 7
2025MoMaGen: Generating Demonstrations under Soft and Hard Constraints for Multi-Step Bimanual Mobile Manipulationcited by 7

19 more

2026RoboAgent: Chaining Basic Capabilities for Embodied Task Planningcited by 6
2025Human-like Navigation in a World Built for Humanscited by 4
2025GraphPad: Inference-Time 3D Scene Graph Updates for Embodied Question Answeringcited by 3
2025AnywhereVLA: Language-Conditioned Exploration and Mobile Manipulationcited by 3
2026OpenFrontier: General Navigation with Visual-Language Grounded Frontierscited by 3
2025Imagine, Verify, Execute: Memory-guided Agentic Exploration with Vision-Language Modelscited by 2
2026From Instruction to Event: Sound-Triggered Mobile Manipulationcited by 2
2026RACAS: Controlling Diverse Robots With a Single Agentic Systemcited by 2
2026ContextFlow: Hierarchical Task-State Alignment for Long-Horizon Embodied Agentscited by 2
2025Casper: Inferring Diverse Intents for Assistive Teleoperation with Vision Language Modelscited by 1
2025A Pragmatist Robot: Learning to Plan Tasks by Experiencing the Real Worldcited by 1
2025Mixed-Initiative Dialog for Human-Robot Collaborative Manipulationcited by 1
2025ROSBag MCP Server: Analyzing Robot Data with LLMs for Agentic Embodied AI Applicationscited by 1
2025Embodied Tree of Thoughts: Deliberate Manipulation Planning with Embodied World Modelcited by 1
2026ARIS: Agentic and Relationship Intelligence System for Social Robotscited by 0
2026When Multi-Robot Systems Meet Agentic AI:Towards Embodied Collective Intelligencecited by 0
2026TypeGo: An OS Runtime for Embodied Agentscited by 0
2026Vision Language Action (VLA) Models for Unmanned Aerial Robotics and Bimanual Manipulation: A Reviewcited by 0
2026UniETP: Unifying Environments for Generalizable Embodied Task Planningcited by 0

Similar, not cited

Close in wording, no citation either way