NavComposer: Composing Language Instructions for Navigation Trajectories through Action-Scene-Object Modularization
Not on the map. Why?
Abstract (arXiv)
Language-guided navigation is a cornerstone of embodied AI, enabling agents to interpret language instructions and navigate complex environments. However, expert-provided instructions are limited in quantity, while synthesized annotations often lack quality, making them insufficient for large-scale research. To address this, we propose NavComposer, a novel framework for automatically generating high-quality navigation instructions. NavComposer explicitly decomposes semantic entities such as actions, scenes, and objects, and recomposes them into natural language instructions. Its modular architecture allows flexible integration of state-of-the-art techniques, while the explicit use of semantic entities enhances both the richness and accuracy of instructions. Moreover, it operates in a data-agnostic manner, supporting adaptation to diverse navigation trajectories without domain-specific training. Complementing NavComposer, we introduce NavInstrCritic, a comprehensive annotation-free evaluation system that assesses navigation instructions on three dimensions: contrastive matching, semantic consistency, and linguistic diversity. NavInstrCritic provides a holistic evaluation of instruction quality, addressing limitations of traditional metrics that rely heavily on expert annotations. By decoupling instruction generation and evaluation from specific navigation agents, our method enables more scalable and generalizable research. Extensive experiments provide direct and practical evidence for the effectiveness of our method.
Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 56 of its 84 references that are in the map. First posted on arXiv 15 Jul 2025.
Builds on
56 of 84 references.
References · 56
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2023Visual Instruction Tuningcited by 7,869
- 2020End-to-End Object Detection with Transformerscited by 7,414
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2024Qwen2.5 Technical Reportcited by 4,942
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
38 more
- 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
- 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,274
- 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,177
- 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
- 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 592
- 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 491
- 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 441
- 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 414
- 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 340
- 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 333
- 2020A Recurrent Vision-and-Language BERT for Navigationcited by 302
- 2021A Survey of Embodied AI: From Simulators to Research Taskscited by 261
- 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 244
- 2019DIODE: A Dense Indoor and Outdoor DEpth Datasetcited by 240
- 2018Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environmentscited by 239
- 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 233
- 2025InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modelingcited by 211
- 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 193
- 2019Vision-Language Navigation with Self-Supervised Auxiliary Reasoning Taskscited by 179
- 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 179
- 2020Improving Vision-and-Language Navigation with Image-Text Pairs from the Webcited by 177
- 2022Bridging the Gap Between Learning in Discrete and Continuous Environments for Vision-and-Language Navigationcited by 132
- 2021Waypoint Models for Instruction-guided Navigation in Continuous Environmentscited by 129
- 2023Scaling Data Generation in Vision-and-Language Navigationcited by 117
- 2022Revisiting Weakly Supervised Pre-Training of Visual Perception Modelscited by 116
- 2022Locally Typical Samplingcited by 111
- 2023Discuss Before Moving: Visual Language Navigation via Multi-expert Discussionscited by 110
- 2022EnvEdit: Environment Editing for Vision-and-Language Navigationcited by 83
- 2021Less is More: Generating Grounded Navigation Instructions from Landmarkscited by 64
- 2016Learning to Navigate the Energy Landscapecited by 62
- 2024GRUtopia: Dream General Robots in a City at Scalecited by 59
- 2024Vision-and-Language Navigation via Causal Learningcited by 51
- 2024Standardizing the Measurement of Text Diversity: A Tool and a Comparative Analysis of Scorescited by 50
- 2023Learning Vision-and-Language Navigation from YouTube Videoscited by 47
- 2023Lana: A Language-Capable Navigator for Instruction Following and Generationcited by 40
- 2023A Review of Deep Learning for Video Captioningcited by 28
- 2023Grounded Entity-Landmark Adaptive Pre-training for Vision-and-Language Navigationcited by 27
- 2020Language-guided Navigation via Cross-Modal Grounding and Alternate Adversarial Learningcited by 22
50 most cited of 56 All 56 (JSON)
Built on by
Citing papers per year: 2025: 2; 2026: 3.
5 papers.
- 2025Rethinking the Embodied Gap in Vision-and-Language Navigation: A Holistic Study of Physical and Visual Disparitiescited by 25
- 2025Aerial Vision-Language Navigation with a Unified Framework for Spatial, Temporal and Embodied Reasoningcited by 8
- 2026A Comprehensive Survey and Systematic Real-World Evaluation of Embodied Vision-and-Language Navigationcited by 1
- 2026GIST: Multimodal Knowledge Extraction and Spatial Grounding via Intelligent Semantic Topologycited by 0
- 2026Query, Align, and Distill: Navigation-Aware Cross-Modal Interaction for Efficient Vision-and-Language Navigationcited by 0
- 2026Query, Align, and Distill: Navigation-Aware Cross-Modal Interaction for Efficient Vision-and-Language Navigation27 Sep 2026
- 2026A Comprehensive Survey and Systematic Real-World Evaluation of Embodied Vision-and-Language Navigation9 Jul 2026
- 2026GIST: Multimodal Knowledge Extraction and Spatial Grounding via Intelligent Semantic Topology16 Apr 2026
- 2025Aerial Vision-Language Navigation with a Unified Framework for Spatial, Temporal and Embodied Reasoning9 Dec 2025
- 2025Rethinking the Embodied Gap in Vision-and-Language Navigation: A Holistic Study of Physical and Visual Disparities17 Jul 2025