mlsanityData to 7 Oct 2026

Global Commander and Local Operative: A Dual-Agent Framework for Scene Navigation

arXiv 2602.18941 ↗First posted 21 Feb 2026cs.CVcited by 0

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

Vision-and-Language Scene navigation is a fundamental capability for embodied human-AI collaboration, requiring agents to follow natural language instructions to execute coherent action sequences in complex environments. Existing approaches either rely on multiple agents, incurring high coordination and resource costs, or adopt a single-agent paradigm, which overloads the agent with both global planning and local perception, often leading to degraded reasoning and instruction drift in long-horizon settings. To address these issues, we introduce DACo, a planning-grounding decoupled architecture that disentangles global deliberation from local grounding. Concretely, it employs a Global Commander for high-level strategic planning and a Local Operative for egocentric observing and fine-grained execution. By disentangling global reasoning from local action, DACo alleviates cognitive overload and improves long-horizon stability. The framework further integrates dynamic subgoal planning and adaptive replanning to enable structured and resilient navigation. Extensive evaluations on R2R, REVERIE, and R4R demonstrate that DACo achieves 4.9%, 6.5%, 5.4% absolute improvements over the best-performing baselines in zero-shot settings, and generalizes effectively across both closed-source (e.g., GPT-4o) and open-source (e.g., Qwen-VL Series) backbones. DACo provides a principled and extensible paradigm for robust long-horizon navigation. Project page: https://github.com/ChocoWu/DACo

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 61 of its 69 references that are in the map. First posted on arXiv 21 Feb 2026.

Builds on

61 of 69 references.

Closest ancestors · 2

Same direction, newest first

  1. 2026OS-Symphony: A Holistic Framework for Robust and Generalist Computer-Using Agent12 Jan
  2. 2025Visual Agentic AI for Spatial Reasoning with a Dynamic API10 Feb

Other references · 59

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  4. 2025Qwen3 Technical Reportcited by 8,195
  5. 2023Visual Instruction Tuningcited by 7,792
  6. 2025Qwen2.5-VL Technical Reportcited by 5,418
  7. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  8. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  9. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
  10. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  11. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  12. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
38 more
  1. 2023Instruction Tuning with GPT-4cited by 676
  2. 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMscited by 424
  3. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 337
  4. 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 330
  5. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 304
  6. 2020A Recurrent Vision-and-Language BERT for Navigationcited by 300
  7. 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Modelscited by 274
  8. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 268
  9. 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 241
  10. 2020Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-trainingcited by 228
  11. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 222
  12. 2024Hierarchical Open-Vocabulary 3D Scene Graphs for Language-Grounded Robot Navigationcited by 195
  13. 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 187
  14. 2019Vision-Language Navigation with Self-Supervised Auxiliary Reasoning Taskscited by 177
  15. 2019Self-Monitoring Navigation Agent via Auxiliary Progress Estimationcited by 173
  16. 2023Towards Learning a Generalist Model for Embodied Navigationcited by 135
  17. 2019Stay on the Path: Instruction Fidelity in Vision-and-Language Navigationcited by 129
  18. 2022Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directionscited by 116
  19. 2023Scaling Data Generation in Vision-and-Language Navigationcited by 115
  20. 2023Discuss Before Moving: Visual Language Navigation via Multi-expert Discussionscited by 108
  21. 2024NavGPT-2: Unleashing Navigational Reasoning Capability for Large Vision-Language Modelscited by 96
  22. 2024MapGPT: Map-Guided Prompting with Adaptive Path Planning for Vision-and-Language Navigationcited by 85
  23. 2025JanusVLN: Decoupling Semantics and Spatiality with Dual Implicit Memory for Vision-Language Navigationcited by 80
  24. 2019Robust Navigation with Language Pretraining and Stochastic Samplingcited by 77
  25. 2020Object-and-Action Aware Model for Visual Language Navigationcited by 77
  26. 2024NavCoT: Boosting LLM-Based Vision-and-Language Navigation via Learning Disentangled Reasoningcited by 77
  27. 2020Evolving Graphical Planner: Contextual Global Planning for Vision-and-Language Navigationcited by 71
  28. 2022HOP: History-and-Order Aware Pre-training for Vision-and-Language Navigationcited by 69
  29. 2021The Road to Know-Where: An Object-and-Room Informed Sequential BERT for Indoor Vision-Language Navigationcited by 60
  30. 2024Visual Prompting in Multimodal Large Language Models: A Surveycited by 52
  31. 2024Vision-and-Language Navigation via Causal Learningcited by 51
  32. 2024Constraint-Aware Zero-Shot Vision-Language Navigation in Continuous Environmentscited by 49
  33. 2023LangNav: Language as a Perceptual Representation for Navigationcited by 47
  34. 2023Fast-Slow Test-Time Adaptation for Online Vision-and-Language Navigationcited by 25
  35. 2020Where Are You? Localization from Embodied Dialogcited by 24
  36. 2024RoomTour3D: Geometry-Aware Video-Instruction Tuning for Embodied Navigationcited by 23
  37. 2024Vinci: A Real-time Embodied Smart Assistant based on Egocentric Vision-Language Modelcited by 19
  38. 2024g3D-LF: Generalizable 3D-Language Feature Fields for Embodied Taskscited by 18

50 most cited of 59 All 59 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Systematic Multi-Agent Vision-and-Language Navigation: Formulation, Benchmark, and Method
  2. 2026A Deployable Embodied Vision-Language Navigation System with Hierarchical Cognition and Context-Aware Exploration ↗
  3. 2026HarnessVLN: Unifying Training-Free Embodied Navigation through an Agent Harness
  4. 2026AgentVLN: Towards Agentic Vision-and-Language Navigation
  5. 2025Run, Ruminate, and Regulate: A Dual-process Thinking System for Vision-and-Language Navigation
  6. 2026Hydra-Nav: Object Navigation via Adaptive Dual-Process Reasoning
  7. 2026Embodied Agents Take Control: Minimal-Interface Zero-Shot Agents Rival Industrial-Scale Policies in Vision-and-Language Navigation
  8. 2025NavForesee: A Unified Vision-Language World Model for Hierarchical Planning and Dual-Horizon Navigation Prediction