# Vision-Language Navigation with Energy-Based Policy

arXiv: https://arxiv.org/abs/2410.14250 · HTML: https://mlsanity.com/paper/2410.14250/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 85 of its 110 references that are in the map.
- First posted on arXiv 18 Oct 2024.

## Builds on (most cited first, 30 of 85)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [LXMERT: Learning Cross-Modality Encoder Representations from Transformers](https://mlsanity.com/paper/1908.07490/) (2019)
- [D4RL: Datasets for Deep Data-Driven Reinforcement Learning](https://mlsanity.com/paper/2004.07219/) (2020)
- [Matterport3D: Learning from RGB-D Data in Indoor Environments](https://mlsanity.com/paper/1709.06158/) (2017)
- [Habitat: A Platform for Embodied AI Research](https://mlsanity.com/paper/1904.01201/) (2019)
- [Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environments](https://mlsanity.com/paper/1711.07280/) (2017)
- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks](https://mlsanity.com/paper/1506.03099/) (2015)
- [Reinforcement Learning with Deep Energy-Based Policies](https://mlsanity.com/paper/1702.08165/) (2017)
- [Model-Based Reinforcement Learning for Atari](https://mlsanity.com/paper/1903.00374/) (2019)
- [Implicit Behavioral Cloning](https://mlsanity.com/paper/2109.00137/) (2021)
- [Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environments](https://mlsanity.com/paper/2004.02857/) (2020)
- [Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Grounding](https://mlsanity.com/paper/2010.07954/) (2020)
- [Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One](https://mlsanity.com/paper/1912.03263/) (2019)
- [Sliced Score Matching: A Scalable Approach to Density and Score Estimation](https://mlsanity.com/paper/1905.07088/) (2019)
- [Speaker-Follower Models for Vision-and-Language Navigation](https://mlsanity.com/paper/1806.02724/) (2018)
- [REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environments](https://mlsanity.com/paper/1904.10151/) (2019)
- [Learning to Explore using Active Neural SLAM](https://mlsanity.com/paper/2004.05155/) (2020)
- [Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation](https://mlsanity.com/paper/1811.10092/) (2018)
- [A Recurrent Vision-and-Language BERT for Navigation](https://mlsanity.com/paper/2011.13922/) (2020)
- [NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Models](https://mlsanity.com/paper/2305.16986/) (2023)
- [Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images](https://mlsanity.com/paper/1506.07365/) (2015)
- [How to Train Your Energy-Based Models](https://mlsanity.com/paper/2101.03288/) (2021)

## Built on by (most cited first, 19 of 19)

- [OctoNav: Towards Generalist Embodied Navigation](https://mlsanity.com/paper/2506.09839/) (2025)
- [g3D-LF: Generalizable 3D-Language Feature Fields for Embodied Tasks](https://mlsanity.com/paper/2411.17030/) (2024)
- [NavRAG: Generating User Demand Instructions for Embodied Navigation through Retrieval-Augmented LLM](https://mlsanity.com/paper/2502.11142/) (2025)
- [MonoDream: Monocular Vision-Language Navigation with Panoramic Dreaming](https://mlsanity.com/paper/2508.02549/) (2025)
- [View Invariant Learning for Vision-Language Navigation in Continuous Environments](https://mlsanity.com/paper/2507.08831/) (2025)
- [NavQ: Learning a Q-Model for Foresighted Vision-and-Language Navigation](https://mlsanity.com/paper/2510.16457/) (2025)
- [History-Enhanced Two-Stage Transformer for Aerial Vision-and-Language Navigation](https://mlsanity.com/paper/2512.14222/) (2025)
- [3D Gaussian Map with Open-Set Semantic Grouping for Vision-Language Navigation](https://mlsanity.com/paper/2605.26500/) (2026)
- [Let's Reward Step-by-Step: Step-Aware Contrastive Alignment for Vision-Language Navigation in Continuous Environments](https://mlsanity.com/paper/2603.09740/) (2026)
- [Does Peer Observation Help? Vision-Sharing Collaboration for Vision-Language Navigation](https://mlsanity.com/paper/2603.20804/) (2026)
- [Goal2Pixel: Grounding Goals to Pixels for Vision-Language Navigation](https://mlsanity.com/paper/2606.01621/) (2026)
- [Towards Next-Generation Healthcare: A Survey of Medical Embodied AI for Perception, Decision-Making, and Action](https://mlsanity.com/paper/2606.15647/) (2026)
- [Continual Adaptation: Environment-Conditional Parameter Generation for Object Detection in Dynamic Scenarios](https://mlsanity.com/paper/2506.24063/) (2025)
- [Text-Scene: A Scene-to-Language Parsing Framework for 3D Scene Understanding](https://mlsanity.com/paper/2509.16721/) (2025)
- [Dynamic Topology Awareness: Breaking the Granularity Rigidity in Vision-Language Navigation](https://mlsanity.com/paper/2601.21751/) (2026)
- [Uncertainty-Aware Gaussian Map for Vision-Language Navigation](https://mlsanity.com/paper/2605.26503/) (2026)
- [PM-Nav: Priori-Map Guided Embodied Navigation in Functional Buildings](https://mlsanity.com/paper/2603.09113/) (2026)
- [Path-level Hindsight Instructions for Semantic Exploration in Vision-Language Navigation](https://mlsanity.com/paper/2607.01754/) (2026)
- [Seek Before You Move: Evidence Seeking for Progress Grounding in Vision-Language Navigation](https://mlsanity.com/paper/2609.37353/) (2026)

## Abstract (arXiv)

Vision-language navigation (VLN) requires an agent to execute actions following human instructions. Existing VLN models are optimized through expert demonstrations by supervised behavioural cloning or incorporating manual reward engineering. While straightforward, these efforts overlook the accumulation of errors in the Markov decision process, and struggle to match the distribution of the expert policy. Going beyond this, we propose an Energy-based Navigation Policy (ENP) to model the joint state-action distribution using an energy-based model. At each step, low energy values correspond to the state-action pairs that the expert is most likely to perform, and vice versa. Theoretically, the optimization objective is equivalent to minimizing the forward divergence between the occupancy measure of the expert and ours. Consequently, ENP learns to globally align with the expert policy by maximizing the likelihood of the actions and modeling the dynamics of the navigation states in a collaborative manner. With a variety of VLN architectures, ENP achieves promising performances on R2R, REVERIE, RxR, and R2R-CE, unleashing the power of existing VLN models.