mlsanityData to 14 Oct 2026

TP-MDDN: Task-Preferenced Multi-Demand-Driven Navigation with Autonomous Decision-Making

arXiv 2511.17225 ↗First posted 21 Nov 2025cs.RO cs.AI cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

In daily life, people often move through spaces to find objects that meet their needs, posing a key challenge in embodied AI. Traditional Demand-Driven Navigation (DDN) handles one need at a time but does not reflect the complexity of real-world tasks involving multiple needs and personal choices. To bridge this gap, we introduce Task-Preferenced Multi-Demand-Driven Navigation (TP-MDDN), a new benchmark for long-horizon navigation involving multiple sub-demands with explicit task preferences. To solve TP-MDDN, we propose AWMSystem, an autonomous decision-making system composed of three key modules: BreakLLM (instruction decomposition), LocateLLM (goal selection), and StatusMLLM (task monitoring). For spatial memory, we design MASMap, which combines 3D point cloud accumulation with 2D semantic mapping for accurate and efficient environmental understanding. Our Dual-Tempo action generation framework integrates zero-shot planning with policy-based fine control, and is further supported by an Adaptive Error Corrector that handles failure cases in real time. Experiments demonstrate that our approach outperforms state-of-the-art baselines in both perception accuracy and navigation robustness.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 64 of its 73 references that are in the map. First posted on arXiv 21 Nov 2025.

Builds on

64 of 73 references.

References · 64

  1. 2024GPT-4o System Cardcited by 4,588
  2. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  3. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,367
  4. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  5. 2024Qwen2 Technical Reportcited by 2,120
  6. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,225
  7. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
  8. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 929
  9. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 869
  10. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 441
  11. 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 380
  12. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 340
38 more
  1. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 305
  2. 2020A Recurrent Vision-and-Language BERT for Navigationcited by 302
  3. 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Modelscited by 280
  4. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 271
  5. 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 244
  6. 2020Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-trainingcited by 231
  7. 2023VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic Navigationcited by 205
  8. 2022CoWs on Pasture: Baselines and Benchmarks for Language-Driven Zero-Shot Object Navigationcited by 194
  9. 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 193
  10. 2024PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMscited by 190
  11. 2020Improving Vision-and-Language Navigation with Image-Text Pairs from the Webcited by 177
  12. 2019Self-Monitoring Navigation Agent via Auxiliary Progress Estimationcited by 176
  13. 2023ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environmentscited by 169
  14. 2024InstructNav: Zero-shot System for Generic Instruction Navigation in Unexplored Environmentcited by 157
  15. 2024Uni-NaVid: A Video-based Vision-Language-Action Model for Unifying Embodied Navigation Taskscited by 156
  16. 2021Airbert: In-domain Pretraining for Vision-and-Language Navigationcited by 130
  17. 2023ESC: Exploration with Soft Commonsense Constraints for Zero-shot Object Navigationcited by 127
  18. 2023GridMM: Grid Memory Map for Vision-and-Language Navigationcited by 124
  19. 2022BEVBert: Multimodal Map Pre-training for Language-guided Navigationcited by 122
  20. 2023Scaling Data Generation in Vision-and-Language Navigationcited by 117
  21. 2023Discuss Before Moving: Visual Language Navigation via Multi-expert Discussionscited by 110
  22. 2024SG-Nav: Online 3D Scene Graph Prompting for LLM-based Zero-shot Object Navigationcited by 109
  23. 2019The Regretful Agent: Heuristic-Aided Navigation through Progress Estimationcited by 105
  24. 2024NavGPT-2: Unleashing Navigational Reasoning Capability for Large Vision-Language Modelscited by 98
  25. 2024MapGPT: Map-Guided Prompting with Adaptive Path Planning for Vision-and-Language Navigationcited by 86
  26. 2018Look Before You Leap: Bridging Model-Free and Model-Based Reinforcement Learning for Planned-Ahead Vision-and-Language Navigationcited by 84
  27. 2022EnvEdit: Environment Editing for Vision-and-Language Navigationcited by 83
  28. 2019Tactical Rewind: Self-Correction via Backtracking in Vision-and-Language Navigationcited by 81
  29. 2020Object-and-Action Aware Model for Visual Language Navigationcited by 77
  30. 2023PanoGen: Text-Conditioned Panoramic Environment Generation for Vision-and-Language Navigationcited by 77
  31. 2024Mobility VLA: Multimodal Instruction Navigation with Long-Context VLMs and Topological Graphscited by 73
  32. 2024Towards Synergistic, Generalized, and Efficient Dual-System for Robotic Manipulationcited by 73
  33. 2023General Object Foundation Model for Images and Videos at Scalecited by 71
  34. 2024OpenFMNav: Towards Open-Set Zero-Shot Object Navigation via Vision-Language Foundation Modelscited by 67
  35. 2024ReMEmbR: Building and Reasoning Over Long-Horizon Spatio-Temporal Memory for Robot Navigationcited by 67
  36. 2024Towards Long-Horizon Vision-Language Navigation: Platform, Benchmark and Methodcited by 67
  37. 2024Lookahead Exploration with Neural Radiance Representation for Continuous Vision-Language Navigationcited by 65
  38. 2023A2Nav: Action-Aware Zero-Shot Robot Navigation by Exploiting Vision-and-Language Ability of Foundation Modelscited by 62

50 most cited of 64 All 64 (JSON)

Built on by

Not cited yet.