Advances in Embodied Navigation Using Large Language Models: A Survey
Not on the map. Why?
Abstract (arXiv)
In recent years, the rapid advancement of Large Language Models (LLMs) such as the Generative Pre-trained Transformer (GPT) has attracted increasing attention due to their potential in a variety of practical applications. The application of LLMs with Embodied Intelligence has emerged as a significant area of focus. Among the myriad applications of LLMs, navigation tasks are particularly noteworthy because they demand a deep understanding of the environment and quick, accurate decision-making. LLMs can augment embodied intelligence systems with sophisticated environmental perception and decision-making support, leveraging their robust language and image-processing capabilities. This article offers an exhaustive summary of the symbiosis between LLMs and embodied intelligence with a focus on navigation. It reviews state-of-the-art models, research methodologies, and assesses the advantages and disadvantages of existing embodied navigation models and datasets. Finally, the article elucidates the role of LLMs in embodied intelligence, based on current research, and forecasts future directions in the field. A comprehensive list of studies in this survey is available at https://github.com/Rongtao-Xu/Awesome-LLM-EN.
Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 71 of its 187 references that are in the map. First posted on arXiv 1 Nov 2023.
Builds on
71 of 187 references.
References · 71
- 2017Attention Is All You Needcited by 59,932
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
- 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
- 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
- 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
38 more
- 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
- 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
- 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
- 2021Grounded Language-Image Pre-trainingcited by 1,147
- 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
- 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 849
- 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
- 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 773
- 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
- 2023A Comprehensive Overview of Large Language Modelscited by 639
- 2022LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Modelscited by 459
- 2022LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Actioncited by 435
- 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 401
- 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 369
- 2022Visual Language Maps for Robot Navigationcited by 362
- 2019DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Framescited by 352
- 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 330
- 2022SQA3D: Situated Question Answering in 3D Scenescited by 303
- 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Modelscited by 274
- 2023PaLI-X: On Scaling up a Multilingual Vision and Language Modelcited by 254
- 2019Vision-and-Dialog Navigationcited by 246
- 2018Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environmentscited by 238
- 2023Grounding Large Language Models in Interactive Environments with Online Reinforcement Learningcited by 193
- 2023LanguageMPC: Large Language Models as Decision Makers for Autonomous Drivingcited by 188
- 2022ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddingscited by 177
- 2023ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environmentscited by 165
- 2022VLP: A Survey on Vision-Language Pre-trainingcited by 159
- 2022Exploration in Deep Reinforcement Learning: A Surveycited by 158
- 2023LLM-Grounder: Open-Vocabulary 3D Visual Grounding with Large Language Model as an Agentcited by 134
- 2023ESC: Exploration with Soft Commonsense Constraints for Zero-shot Object Navigationcited by 126
- 2021SOON: Scenario Oriented Object Navigation with Graph-based Explorationcited by 118
- 2023L3MVN: Leveraging Large Language Models for Visual Target Navigationcited by 118
- 2022BEVBert: Multimodal Map Pre-training for Language-guided Navigationcited by 117
- 2018Vision-based Navigation with Language-based Assistance via Imitation Learning with Indirect Interventioncited by 77
- 2020ViNG: Learning Open-World Navigation with Visual Goalscited by 77
- 2023OVRL-V2: A simple state-of-art baseline for ImageNav and ObjectNavcited by 68
- 2023VELMA: Verbalization Embodiment of LLM Agents for Vision and Language Navigation in Street Viewcited by 67
- 2022Synthetic Data -- what, why and how?cited by 66
50 most cited of 71 All 71 (JSON)
Built on by
Citing papers per year: 2024: 2; 2025: 8; 2026: 4.
14 papers.
Sort:
By area:
- 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learningcited by 357
- 2025Multimodal Fusion and Vision-Language Models: A Survey for Robot Visioncited by 37
- 2025Spatial Reasoning in Multimodal Large Language Models: A Survey of Tasks, Benchmarks and Methodscited by 22
- 2025A Survey of Large Language Model-Powered Spatial Intelligence Across Scales: Advances in Embodied Agents, Smart Cities, and Earth Sciencecited by 11
- 2025From Text to Space: Mapping Abstract Spatial Models in LLMs during a Grid-World Navigation Taskcited by 10
- 2025Enhancing Text-to-Image Diffusion Transformer via Split-Text Conditioningcited by 7
- 2024MLIP: Efficient Multi-Perspective Language-Image Pretraining with Exhaustive Data Utilizationcited by 6
- 2025ReEXplore: Improving MLLMs for Embodied Exploration with Contextualized Retrospective Experience Replaycited by 4
- 2024Personalized Embodied Navigation for Portable Object Findingcited by 3
- 2026MosaicThinker: On-Device Visual Spatial Reasoning for Embodied AI via Iterative Construction of Space Representationcited by 2
- 2025Boosting Adversarial Transferability via Commonality-Oriented Gradient Optimizationcited by 1
- 2026ReasonNavi: Human-Inspired Global Map Reasoning for Zero-Shot Embodied Navigationcited by 1
- 2026Speaking the Navigator's Language: Trajectory-Grounded Instruction Translation for Frozen Aerial VLN Agents7 Oct 2026
- 2026A Comprehensive Survey and Systematic Real-World Evaluation of Embodied Vision-and-Language Navigation9 Jul 2026
- 2026MosaicThinker: On-Device Visual Spatial Reasoning for Embodied AI via Iterative Construction of Space Representation6 Feb 2026
- 2026ReasonNavi: Human-Inspired Global Map Reasoning for Zero-Shot Embodied Navigation26 Jan 2026
- 2025ReEXplore: Improving MLLMs for Embodied Exploration with Contextualized Retrospective Experience Replay24 Nov 2025
- 2025Spatial Reasoning in Multimodal Large Language Models: A Survey of Tasks, Benchmarks and Methods14 Nov 2025
- 2025Boosting Adversarial Transferability via Commonality-Oriented Gradient Optimization8 Jun 2025
- 2025Enhancing Text-to-Image Diffusion Transformer via Split-Text Conditioning25 May 2025
- 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning24 Apr 2025
- 2025A Survey of Large Language Model-Powered Spatial Intelligence Across Scales: Advances in Embodied Agents, Smart Cities, and Earth Science14 Apr 2025
- 2025Multimodal Fusion and Vision-Language Models: A Survey for Robot Vision3 Apr 2025
- 2025From Text to Space: Mapping Abstract Spatial Models in LLMs during a Grid-World Navigation Task23 Feb 2025
- 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learningcited by 357
- 2025From Text to Space: Mapping Abstract Spatial Models in LLMs during a Grid-World Navigation Taskcited by 10
- 2026Speaking the Navigator's Language: Trajectory-Grounded Instruction Translation for Frozen Aerial VLN Agentscited by 0
- 2026Speaking the Navigator's Language: Trajectory-Grounded Instruction Translation for Frozen Aerial VLN Agents7 Oct 2026
- 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning24 Apr 2025
- 2025From Text to Space: Mapping Abstract Spatial Models in LLMs during a Grid-World Navigation Task23 Feb 2025
- 2025Spatial Reasoning in Multimodal Large Language Models: A Survey of Tasks, Benchmarks and Methodscited by 22
- 2026MosaicThinker: On-Device Visual Spatial Reasoning for Embodied AI via Iterative Construction of Space Representationcited by 2
- 2026ReasonNavi: Human-Inspired Global Map Reasoning for Zero-Shot Embodied Navigationcited by 1
- 2026MosaicThinker: On-Device Visual Spatial Reasoning for Embodied AI via Iterative Construction of Space Representation6 Feb 2026
- 2026ReasonNavi: Human-Inspired Global Map Reasoning for Zero-Shot Embodied Navigation26 Jan 2026
- 2025Spatial Reasoning in Multimodal Large Language Models: A Survey of Tasks, Benchmarks and Methods14 Nov 2025
- 2024MLIP: Efficient Multi-Perspective Language-Image Pretraining with Exhaustive Data Utilizationcited by 6
- 2025ReEXplore: Improving MLLMs for Embodied Exploration with Contextualized Retrospective Experience Replaycited by 4
- 2024Personalized Embodied Navigation for Portable Object Findingcited by 3
- 2025Boosting Adversarial Transferability via Commonality-Oriented Gradient Optimizationcited by 1
- 2026A Comprehensive Survey and Systematic Real-World Evaluation of Embodied Vision-and-Language Navigationcited by 1
- 2026A Comprehensive Survey and Systematic Real-World Evaluation of Embodied Vision-and-Language Navigation9 Jul 2026
- 2025ReEXplore: Improving MLLMs for Embodied Exploration with Contextualized Retrospective Experience Replay24 Nov 2025
- 2025Boosting Adversarial Transferability via Commonality-Oriented Gradient Optimization8 Jun 2025
- 2024MLIP: Efficient Multi-Perspective Language-Image Pretraining with Exhaustive Data Utilization3 Jun 2024
- 2024Personalized Embodied Navigation for Portable Object Finding14 Mar 2024