mlsanityData to 14 Oct 2026

ImagineNav++: Prompting Vision-Language Models as Embodied Navigator through Scene Imagination

arXiv 2512.17435 ↗First posted 19 Dec 2025cs.ROcited by 1

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Visual navigation is a fundamental capability for autonomous home-assistance robots, enabling long-horizon tasks such as object search. While recent methods have leveraged Large Language Models (LLMs) to incorporate commonsense reasoning and improve exploration efficiency, their planning remains constrained by textual representations, which cannot adequately capture spatial occupancy or scene geometry--critical factors for navigation decisions. We explore whether Vision-Language Models (VLMs) can achieve mapless visual navigation using only onboard RGB/RGB-D streams, unlocking their potential for spatial perception and planning. We achieve this through an imagination-powered navigation framework, ImagineNav++, which imagines future observation images from candidate robot views and translates navigation planning into a simple best-view image selection problem for VLMs. First, a future-view imagination module distills human navigation preferences to generate semantically meaningful viewpoints with high exploration potential. These imagined views then serve as visual prompts for the VLM to identify the most informative viewpoint. To maintain spatial consistency, we develop a selective foveation memory mechanism, which hierarchically integrates keyframe observations via a sparse-to-dense framework, constructing a compact yet comprehensive memory for long-term spatial reasoning. This approach transforms goal-oriented navigation into a series of tractable point-goal navigation tasks. Extensive experiments on open-vocabulary object and instance navigation benchmarks show that ImagineNav++ achieves SOTA performance in mapless settings, even surpassing most map-based methods, highlighting the importance of scene imagination and memory in VLM-based spatial reasoning.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 89 of its 101 references that are in the map. First posted on arXiv 19 Dec 2025.

Builds on

89 of 101 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025Mem2Ego: Empowering Vision-Language Models with Global-to-Ego Memory for Long-Horizon Embodied Navigation20 Feb
  2. 2024End-to-End Navigation with Vision Language Models: Transforming Spatial Reasoning into Question-Answering8 Nov
  3. 2024ImagineNav: Prompting Vision-Language Models as Embodied Navigator through Scene Imagination13 Oct
  4. 2024OpenFMNav: Towards Open-Set Zero-Shot Object Navigation via Vision-Language Foundation Models16 Feb
  5. 2024PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMs12 Feb
  6. 2023VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic Navigation6 Dec
  7. 2023Multimodal Large Language Model for Visual Navigation12 Oct
  8. 2022Visual Language Maps for Robot Navigation11 Oct

Other references · 81

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2023GPT-4 Technical Reportcited by 18,396
  5. 2017Proximal Policy Optimization Algorithmscited by 15,215
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  7. 2023Segment Anythingcited by 8,234
  8. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  9. 2023Visual Instruction Tuningcited by 7,869
  10. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  11. 2017Mask R-CNNcited by 7,277
  12. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
38 more
  1. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  2. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  3. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  4. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  5. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  6. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  7. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  8. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  9. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,210
  10. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,035
  11. 2023VILA: On Pre-training for Visual Language Modelscited by 692
  12. 2022Detecting Twenty-thousand Classes using Image-level Supervisioncited by 614
  13. 2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Modelcited by 602
  14. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 592
  15. 2018On Evaluation of Embodied Navigation Agentscited by 547
  16. 2022Socratic Models: Composing Zero-Shot Multimodal Reasoning with Languagecited by 513
  17. 2024YOLO-World: Real-Time Open-Vocabulary Object Detectioncited by 503
  18. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 481
  19. 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 429
  20. 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Modelscited by 280
  21. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 233
  22. 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 228
  23. 2022CoWs on Pasture: Baselines and Benchmarks for Language-Driven Zero-Shot Object Navigationcited by 194
  24. 2021Simple but Effective: CLIP Embeddings for Embodied AIcited by 191
  25. 2022ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddingscited by 178
  26. 2023Habitat Synthetic Scenes Dataset (HSSD-200): An Analysis of 3D Scene Scale and Realism Tradeoffs for ObjectGoal Navigationcited by 167
  27. 2024Uni-NaVid: A Video-based Vision-Language-Action Model for Unifying Embodied Navigation Taskscited by 156
  28. 2022PONI: Potential Functions for ObjectGoal Navigation with Interaction-free Learningcited by 150
  29. 2023ESC: Exploration with Soft Commonsense Constraints for Zero-shot Object Navigationcited by 127
  30. 2023GridMM: Grid Memory Map for Vision-and-Language Navigationcited by 124
  31. 2023L3MVN: Leveraging Large Language Models for Visual Target Navigationcited by 120
  32. 2022Habitat-Web: Learning Embodied Object-Search Strategies from Human Demonstrations at Scalecited by 117
  33. 2023Consistent View Synthesis with Pose-Guided Diffusion Modelscited by 117
  34. 2023Instruct2Act: Mapping Multi-modality Instructions to Robotic Actions with Large Language Modelcited by 116
  35. 2023ZeroNVS: Zero-Shot 360-Degree View Synthesis from a Single Imagecited by 111
  36. 2024SG-Nav: Online 3D Scene Graph Prompting for LLM-based Zero-shot Object Navigationcited by 109
  37. 2020Occupancy Anticipation for Efficient Exploration and Navigationcited by 106
  38. 2024CoPa: General Robotic Manipulation through Spatial Constraints of Parts with Foundation Modelscited by 99

50 most cited of 81 All 81 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2025VISTA: Generative Visual Imagination for Vision-and-Language Navigation
  2. 2025A Navigation Framework Utilizing Vision-Language Models
  3. 2026ImagiNav: Scalable Embodied Navigation via Generative Visual Prediction and Inverse Dynamics
  4. 2026FantasyVLN: Unified Multimodal Chain-of-Thought Reasoning for Vision-Language Navigation
  5. 2025VISTAv2: World Imagination for Indoor Vision-and-Language Navigation
  6. 2026GemNav: Discrete-Token Visual Robot Navigation using a Multimodal Large Language Model
  7. 2025Imagine, Verify, Execute: Memory-guided Agentic Exploration with Vision-Language Models
  8. 2026AgentVLN: Towards Agentic Vision-and-Language Navigation