mlsanityData to 7 Oct 2026

Advances in Embodied Navigation Using Large Language Models: A Survey

arXiv 2311.00530 ↗First posted 1 Nov 2023cs.AIcited by 14

Not on the map. Why?

Abstract (arXiv)

In recent years, the rapid advancement of Large Language Models (LLMs) such as the Generative Pre-trained Transformer (GPT) has attracted increasing attention due to their potential in a variety of practical applications. The application of LLMs with Embodied Intelligence has emerged as a significant area of focus. Among the myriad applications of LLMs, navigation tasks are particularly noteworthy because they demand a deep understanding of the environment and quick, accurate decision-making. LLMs can augment embodied intelligence systems with sophisticated environmental perception and decision-making support, leveraging their robust language and image-processing capabilities. This article offers an exhaustive summary of the symbiosis between LLMs and embodied intelligence with a focus on navigation. It reviews state-of-the-art models, research methodologies, and assesses the advantages and disadvantages of existing embodied navigation models and datasets. Finally, the article elucidates the role of LLMs in embodied intelligence, based on current research, and forecasts future directions in the field. A comprehensive list of studies in this survey is available at https://github.com/Rongtao-Xu/Awesome-LLM-EN.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 71 of its 187 references that are in the map. First posted on arXiv 1 Nov 2023.

Builds on

71 of 187 references.

References · 71

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2020Language Models are Few-Shot Learnerscited by 32,199
  6. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  7. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  8. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  9. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  10. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  11. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  12. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
38 more
  1. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  2. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  3. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  4. 2021Grounded Language-Image Pre-trainingcited by 1,147
  5. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  6. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 849
  7. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  8. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 773
  9. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
  10. 2023A Comprehensive Overview of Large Language Modelscited by 639
  11. 2022LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Modelscited by 459
  12. 2022LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Actioncited by 435
  13. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 401
  14. 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 369
  15. 2022Visual Language Maps for Robot Navigationcited by 362
  16. 2019DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Framescited by 352
  17. 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 330
  18. 2022SQA3D: Situated Question Answering in 3D Scenescited by 303
  19. 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Modelscited by 274
  20. 2023PaLI-X: On Scaling up a Multilingual Vision and Language Modelcited by 254
  21. 2019Vision-and-Dialog Navigationcited by 246
  22. 2018Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environmentscited by 238
  23. 2023Grounding Large Language Models in Interactive Environments with Online Reinforcement Learningcited by 193
  24. 2023LanguageMPC: Large Language Models as Decision Makers for Autonomous Drivingcited by 188
  25. 2022ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddingscited by 177
  26. 2023ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environmentscited by 165
  27. 2022VLP: A Survey on Vision-Language Pre-trainingcited by 159
  28. 2022Exploration in Deep Reinforcement Learning: A Surveycited by 158
  29. 2023LLM-Grounder: Open-Vocabulary 3D Visual Grounding with Large Language Model as an Agentcited by 134
  30. 2023ESC: Exploration with Soft Commonsense Constraints for Zero-shot Object Navigationcited by 126
  31. 2021SOON: Scenario Oriented Object Navigation with Graph-based Explorationcited by 118
  32. 2023L3MVN: Leveraging Large Language Models for Visual Target Navigationcited by 118
  33. 2022BEVBert: Multimodal Map Pre-training for Language-guided Navigationcited by 117
  34. 2018Vision-based Navigation with Language-based Assistance via Imitation Learning with Indirect Interventioncited by 77
  35. 2020ViNG: Learning Open-World Navigation with Visual Goalscited by 77
  36. 2023OVRL-V2: A simple state-of-art baseline for ImageNav and ObjectNavcited by 68
  37. 2023VELMA: Verbalization Embodiment of LLM Agents for Vision and Language Navigation in Street Viewcited by 67
  38. 2022Synthetic Data -- what, why and how?cited by 66

50 most cited of 71 All 71 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 8; 2026: 4.

14 papers.

Sort:

By area:

  1. 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learningcited by 357
  2. 2025Multimodal Fusion and Vision-Language Models: A Survey for Robot Visioncited by 37
  3. 2025Spatial Reasoning in Multimodal Large Language Models: A Survey of Tasks, Benchmarks and Methodscited by 22
  4. 2025A Survey of Large Language Model-Powered Spatial Intelligence Across Scales: Advances in Embodied Agents, Smart Cities, and Earth Sciencecited by 11
  5. 2025From Text to Space: Mapping Abstract Spatial Models in LLMs during a Grid-World Navigation Taskcited by 10
  6. 2025Enhancing Text-to-Image Diffusion Transformer via Split-Text Conditioningcited by 7
  7. 2024MLIP: Efficient Multi-Perspective Language-Image Pretraining with Exhaustive Data Utilizationcited by 6
  8. 2025ReEXplore: Improving MLLMs for Embodied Exploration with Contextualized Retrospective Experience Replaycited by 4
  9. 2024Personalized Embodied Navigation for Portable Object Findingcited by 3
  10. 2026MosaicThinker: On-Device Visual Spatial Reasoning for Embodied AI via Iterative Construction of Space Representationcited by 2
  11. 2025Boosting Adversarial Transferability via Commonality-Oriented Gradient Optimizationcited by 1
  12. 2026ReasonNavi: Human-Inspired Global Map Reasoning for Zero-Shot Embodied Navigationcited by 1
2 more
  1. 2026A Comprehensive Survey and Systematic Real-World Evaluation of Embodied Vision-and-Language Navigationcited by 1
  2. 2026Speaking the Navigator's Language: Trajectory-Grounded Instruction Translation for Frozen Aerial VLN Agentscited by 0