Global Commander and Local Operative: A Dual-Agent Framework for Scene Navigation
On the map: LLM agents and tool use → Multimodal and embodied agents
Abstract (arXiv)
Vision-and-Language Scene navigation is a fundamental capability for embodied human-AI collaboration, requiring agents to follow natural language instructions to execute coherent action sequences in complex environments. Existing approaches either rely on multiple agents, incurring high coordination and resource costs, or adopt a single-agent paradigm, which overloads the agent with both global planning and local perception, often leading to degraded reasoning and instruction drift in long-horizon settings. To address these issues, we introduce DACo, a planning-grounding decoupled architecture that disentangles global deliberation from local grounding. Concretely, it employs a Global Commander for high-level strategic planning and a Local Operative for egocentric observing and fine-grained execution. By disentangling global reasoning from local action, DACo alleviates cognitive overload and improves long-horizon stability. The framework further integrates dynamic subgoal planning and adaptive replanning to enable structured and resilient navigation. Extensive evaluations on R2R, REVERIE, and R4R demonstrate that DACo achieves 4.9%, 6.5%, 5.4% absolute improvements over the best-performing baselines in zero-shot settings, and generalizes effectively across both closed-source (e.g., GPT-4o) and open-source (e.g., Qwen-VL Series) backbones. DACo provides a principled and extensible paradigm for robust long-horizon navigation. Project page: https://github.com/ChocoWu/DACo
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 61 of its 69 references that are in the map. First posted on arXiv 21 Feb 2026.
Builds on
61 of 69 references.
Closest ancestors · 2
Same direction, newest first
- 2026OS-Symphony: A Holistic Framework for Robust and Generalist Computer-Using Agent12 Jan
- 2025Visual Agentic AI for Spatial Reasoning with a Dynamic API10 Feb
Other references · 59
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2025Qwen3 Technical Reportcited by 8,195
- 2023Visual Instruction Tuningcited by 7,792
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
- 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
- 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
- 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
- 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
38 more
- 2023Instruction Tuning with GPT-4cited by 676
- 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMscited by 424
- 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 337
- 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 330
- 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 304
- 2020A Recurrent Vision-and-Language BERT for Navigationcited by 300
- 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Modelscited by 274
- 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 268
- 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 241
- 2020Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-trainingcited by 228
- 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 222
- 2024Hierarchical Open-Vocabulary 3D Scene Graphs for Language-Grounded Robot Navigationcited by 195
- 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 187
- 2019Vision-Language Navigation with Self-Supervised Auxiliary Reasoning Taskscited by 177
- 2019Self-Monitoring Navigation Agent via Auxiliary Progress Estimationcited by 173
- 2023Towards Learning a Generalist Model for Embodied Navigationcited by 135
- 2019Stay on the Path: Instruction Fidelity in Vision-and-Language Navigationcited by 129
- 2022Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directionscited by 116
- 2023Scaling Data Generation in Vision-and-Language Navigationcited by 115
- 2023Discuss Before Moving: Visual Language Navigation via Multi-expert Discussionscited by 108
- 2024NavGPT-2: Unleashing Navigational Reasoning Capability for Large Vision-Language Modelscited by 96
- 2024MapGPT: Map-Guided Prompting with Adaptive Path Planning for Vision-and-Language Navigationcited by 85
- 2025JanusVLN: Decoupling Semantics and Spatiality with Dual Implicit Memory for Vision-Language Navigationcited by 80
- 2019Robust Navigation with Language Pretraining and Stochastic Samplingcited by 77
- 2020Object-and-Action Aware Model for Visual Language Navigationcited by 77
- 2024NavCoT: Boosting LLM-Based Vision-and-Language Navigation via Learning Disentangled Reasoningcited by 77
- 2020Evolving Graphical Planner: Contextual Global Planning for Vision-and-Language Navigationcited by 71
- 2022HOP: History-and-Order Aware Pre-training for Vision-and-Language Navigationcited by 69
- 2021The Road to Know-Where: An Object-and-Room Informed Sequential BERT for Indoor Vision-Language Navigationcited by 60
- 2024Visual Prompting in Multimodal Large Language Models: A Surveycited by 52
- 2024Vision-and-Language Navigation via Causal Learningcited by 51
- 2024Constraint-Aware Zero-Shot Vision-Language Navigation in Continuous Environmentscited by 49
- 2023LangNav: Language as a Perceptual Representation for Navigationcited by 47
- 2023Fast-Slow Test-Time Adaptation for Online Vision-and-Language Navigationcited by 25
- 2020Where Are You? Localization from Embodied Dialogcited by 24
- 2024RoomTour3D: Geometry-Aware Video-Instruction Tuning for Embodied Navigationcited by 23
- 2024Vinci: A Real-time Embodied Smart Assistant based on Egocentric Vision-Language Modelcited by 19
- 2024g3D-LF: Generalizable 3D-Language Feature Fields for Embodied Taskscited by 18
50 most cited of 59 All 59 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Systematic Multi-Agent Vision-and-Language Navigation: Formulation, Benchmark, and Method
- 2026A Deployable Embodied Vision-Language Navigation System with Hierarchical Cognition and Context-Aware Exploration ↗
- 2026HarnessVLN: Unifying Training-Free Embodied Navigation through an Agent Harness
- 2026AgentVLN: Towards Agentic Vision-and-Language Navigation
- 2025Run, Ruminate, and Regulate: A Dual-process Thinking System for Vision-and-Language Navigation
- 2026Hydra-Nav: Object Navigation via Adaptive Dual-Process Reasoning
- 2026Embodied Agents Take Control: Minimal-Interface Zero-Shot Agents Rival Industrial-Scale Policies in Vision-and-Language Navigation
- 2025NavForesee: A Unified Vision-Language World Model for Hierarchical Planning and Dual-Horizon Navigation Prediction