mlsanityData to 14 Oct 2026

NavComposer: Composing Language Instructions for Navigation Trajectories through Action-Scene-Object Modularization

arXiv 2507.10894 ↗First posted 15 Jul 2025cs.AI cs.CL cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

Language-guided navigation is a cornerstone of embodied AI, enabling agents to interpret language instructions and navigate complex environments. However, expert-provided instructions are limited in quantity, while synthesized annotations often lack quality, making them insufficient for large-scale research. To address this, we propose NavComposer, a novel framework for automatically generating high-quality navigation instructions. NavComposer explicitly decomposes semantic entities such as actions, scenes, and objects, and recomposes them into natural language instructions. Its modular architecture allows flexible integration of state-of-the-art techniques, while the explicit use of semantic entities enhances both the richness and accuracy of instructions. Moreover, it operates in a data-agnostic manner, supporting adaptation to diverse navigation trajectories without domain-specific training. Complementing NavComposer, we introduce NavInstrCritic, a comprehensive annotation-free evaluation system that assesses navigation instructions on three dimensions: contrastive matching, semantic consistency, and linguistic diversity. NavInstrCritic provides a holistic evaluation of instruction quality, addressing limitations of traditional metrics that rely heavily on expert annotations. By decoupling instruction generation and evaluation from specific navigation agents, our method enables more scalable and generalizable research. Extensive experiments provide direct and practical evidence for the effectiveness of our method.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 56 of its 84 references that are in the map. First posted on arXiv 15 Jul 2025.

Builds on

56 of 84 references.

References · 56

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  5. 2023Visual Instruction Tuningcited by 7,869
  6. 2020End-to-End Object Detection with Transformerscited by 7,414
  7. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  8. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  9. 2025Qwen2.5-VL Technical Reportcited by 5,587
  10. 2024Qwen2.5 Technical Reportcited by 4,942
  11. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  12. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
38 more
  1. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  2. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,274
  3. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,177
  4. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
  5. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 592
  6. 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 491
  7. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 441
  8. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 414
  9. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 340
  10. 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 333
  11. 2020A Recurrent Vision-and-Language BERT for Navigationcited by 302
  12. 2021A Survey of Embodied AI: From Simulators to Research Taskscited by 261
  13. 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 244
  14. 2019DIODE: A Dense Indoor and Outdoor DEpth Datasetcited by 240
  15. 2018Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environmentscited by 239
  16. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 233
  17. 2025InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modelingcited by 211
  18. 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 193
  19. 2019Vision-Language Navigation with Self-Supervised Auxiliary Reasoning Taskscited by 179
  20. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 179
  21. 2020Improving Vision-and-Language Navigation with Image-Text Pairs from the Webcited by 177
  22. 2022Bridging the Gap Between Learning in Discrete and Continuous Environments for Vision-and-Language Navigationcited by 132
  23. 2021Waypoint Models for Instruction-guided Navigation in Continuous Environmentscited by 129
  24. 2023Scaling Data Generation in Vision-and-Language Navigationcited by 117
  25. 2022Revisiting Weakly Supervised Pre-Training of Visual Perception Modelscited by 116
  26. 2022Locally Typical Samplingcited by 111
  27. 2023Discuss Before Moving: Visual Language Navigation via Multi-expert Discussionscited by 110
  28. 2022EnvEdit: Environment Editing for Vision-and-Language Navigationcited by 83
  29. 2021Less is More: Generating Grounded Navigation Instructions from Landmarkscited by 64
  30. 2016Learning to Navigate the Energy Landscapecited by 62
  31. 2024GRUtopia: Dream General Robots in a City at Scalecited by 59
  32. 2024Vision-and-Language Navigation via Causal Learningcited by 51
  33. 2024Standardizing the Measurement of Text Diversity: A Tool and a Comparative Analysis of Scorescited by 50
  34. 2023Learning Vision-and-Language Navigation from YouTube Videoscited by 47
  35. 2023Lana: A Language-Capable Navigator for Instruction Following and Generationcited by 40
  36. 2023A Review of Deep Learning for Video Captioningcited by 28
  37. 2023Grounded Entity-Landmark Adaptive Pre-training for Vision-and-Language Navigationcited by 27
  38. 2020Language-guided Navigation via Cross-Modal Grounding and Alternate Adversarial Learningcited by 22

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 3.

5 papers.