mlsanityData to 7 Oct 2026

Distilling LLM Prior to Flow Model for Generalizable Agent's Imagination in Object Goal Navigation

arXiv 2508.09423 ↗First posted 13 Aug 2025cs.CV cs.ROcited by 3

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

The Object Goal Navigation (ObjectNav) task challenges agents to locate a specified object in an unseen environment by imagining unobserved regions of the scene. Prior approaches rely on deterministic and discriminative models to complete semantic maps, overlooking the inherent uncertainty in indoor layouts and limiting their ability to generalize to unseen environments. In this work, we propose GOAL, a generative flow-based framework that models the semantic distribution of indoor environments by bridging observed regions with LLM-enriched full-scene semantic maps. During training, spatial priors inferred from large language models (LLMs) are encoded as two-dimensional Gaussian fields and injected into target maps, distilling rich contextual knowledge into the flow model and enabling more generalizable completions. Extensive experiments demonstrate that GOAL achieves state-of-the-art performance on MP3D and Gibson, and shows strong generalization in transfer settings to HM3D. Codes and pretrained models are available at https://github.com/Badi-Li/GOAL.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 71 of its 80 references that are in the map. First posted on arXiv 13 Aug 2025.

Builds on

71 of 80 references.

References · 71

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2023GPT-4 Technical Reportcited by 18,132
  7. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  8. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  9. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  10. 2017Mask R-CNNcited by 7,259
  11. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  12. 2022Scalable Diffusion Models with Transformerscited by 5,931
38 more
  1. 2022Flow Matching for Generative Modelingcited by 5,426
  2. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  3. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  4. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  5. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  6. 2023Consistency Modelscited by 1,778
  7. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  8. 2020Point Transformercited by 1,462
  9. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,419
  10. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  11. 2021Palette: Image-to-Image Diffusion Modelscited by 1,296
  12. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  13. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
  14. 2023Improving and generalizing flow-based generative models with minibatch optimal transportcited by 835
  15. 20173D Semantic Segmentation with Submanifold Sparse Convolutional Networkscited by 746
  16. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 599
  17. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
  18. 2023Point Transformer V3: Simpler, Faster, Strongercited by 579
  19. 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 501
  20. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 477
  21. 2023Improved Techniques for Training Consistency Modelscited by 430
  22. 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 421
  23. 2022Point Transformer V2: Grouped Vector Attention and Partition-based Poolingcited by 384
  24. 2024One Step Diffusion via Shortcut Modelscited by 373
  25. 2024Simplifying, Stabilizing and Scaling Continuous-Time Consistency Modelscited by 276
  26. 2024Navigation World Modelscited by 262
  27. 2023I2SB: Image-to-Image Schr"odinger Bridgecited by 214
  28. 2022CoWs on Pasture: Baselines and Benchmarks for Language-Driven Zero-Shot Object Navigationcited by 194
  29. 2022ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddingscited by 177
  30. 2022PONI: Potential Functions for ObjectGoal Navigation with Interaction-free Learningcited by 149
  31. 2018Visual Semantic Navigation using Scene Priorscited by 127
  32. 2023ESC: Exploration with Soft Commonsense Constraints for Zero-shot Object Navigationcited by 126
  33. 2023L3MVN: Leveraging Large Language Models for Visual Target Navigationcited by 118
  34. 2022Habitat-Web: Learning Embodied Object-Search Strategies from Human Demonstrations at Scalecited by 116
  35. 2024SG-Nav: Online 3D Scene Graph Prompting for LLM-based Zero-shot Object Navigationcited by 106
  36. 2023Stochastic interpolants with data-dependent couplingscited by 94
  37. 2018Learning to Learn How to Learn: Self-Adaptive Visual Navigation Using Meta-Learningcited by 91
  38. 2023Bridging Zero-shot Object Navigation and Foundation Models through Pixel-Guided Navigation Skillcited by 75

50 most cited of 71 All 71 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2024Cognitive Planning for Object Goal Navigation using Generative AI Models
  2. 2026HRO: Hierarchical Room-to-Object Framework for Zero-Shot Object Goal Navigation with Large Language Models
  3. 2024Diffusion as Reasoning: Enhancing Object Navigation via Diffusion Model Conditioned on LLM-based Object-Room Knowledge
  4. 2026From Reactive to Map-Based AI: Tuned Local LLMs for Semantic Zone Inference in Object-Goal Navigation
  5. 2026GoalVLM: VLM-driven Object Goal Navigation for Multi-Agent System
  6. 2026LangMap: A Human-Verified Benchmark for Hierarchical Open-Vocabulary Goal Navigation
  7. 2025VISTA: Generative Visual Imagination for Vision-and-Language Navigation
  8. 2025osmAG-LLM: Zero-Shot Open-Vocabulary Object Navigation via Semantic Maps and Large Language Models Reasoning