Occupancy Anticipation for Efficient Exploration and Navigation
Not on the map. Why?
Abstract (arXiv)
State-of-the-art navigation methods leverage a spatial memory to generalize to new environments, but their occupancy maps are limited to capturing the geometric structures directly observed by the agent. We propose occupancy anticipation, where the agent uses its egocentric RGB-D observations to infer the occupancy state beyond the visible regions. In doing so, the agent builds its spatial awareness more rapidly, which facilitates efficient exploration and navigation in 3D environments. By exploiting context in both the egocentric views and top-down maps our model successfully anticipates a broader map of the environment, with performance significantly better than strong baselines. Furthermore, when deployed for the sequential decision-making tasks of exploration and navigation, our model outperforms state-of-the-art methods on the Gibson and Matterport3D datasets. Our approach is the winning entry in the 2020 Habitat PointNav Challenge. Project page: http://vision.cs.utexas.edu/projects/occupancy_anticipation/
Key facts. Cited by 104 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Aug 2020.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 26; 2022: 29; 2023: 14; 2024: 12; 2025: 15; 2026: 8.
104 papers.
Sort:
By area:
- 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
- 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 501
- 2021A Survey of Embodied AI: From Simulators to Research Taskscited by 256
- 2022PONI: Potential Functions for ObjectGoal Navigation with Interaction-free Learningcited by 149
- 2022Navigating to Objects in the Real Worldcited by 104
- 2021Visual Room Rearrangementcited by 101
- 2024GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigationcited by 93
- 2022Cross-modal Map Learning for Vision and Language Navigationcited by 91
- 2022Weakly-Supervised Multi-Granularity Map Learning for Vision-and-Language Navigationcited by 86
- 2022Zero Experience Required: Plug & Play Modular Transfer Learning for Semantic Visual Navigationcited by 66
- 2021The ThreeDWorld Transport Challenge: A Visually Guided Task-and-Motion Planning Benchmark for Physically Realistic Embodied AIcited by 64
- 2021No RL, No Simulation: Learning to Navigate without Navigatingcited by 63
38 more
- 2021RobustNav: Towards Benchmarking Robustness in Embodied Navigationcited by 52
- 2023Renderable Neural Radiance Map for Visual Navigationcited by 51
- 2021Learning to Map for Active Semantic Goal Navigationcited by 47
- 2022Retrospectives on the Embodied AI Workshopcited by 46
- 2021Move2Hear: Active Audio-Visual Source Separationcited by 35
- 2021Differentiable SLAM-net: Learning Particle SLAM for Visual Navigationcited by 34
- 2022Is Mapping Necessary for Realistic PointGoal Navigation?cited by 32
- 2022Uncertainty-driven Planner for Exploration and Navigationcited by 31
- 2022TIDEE: Tidying Up Novel Rooms using Visuo-Semantic Commonsense Priorscited by 30
- 2023Active Neural Mappingcited by 30
- 2021The Surprising Effectiveness of Visual Odometry Techniques for Embodied PointGoal Navigationcited by 27
- 2024NARUTO: Neural Active Reconstruction from Uncertain Target Observationscited by 27
- 2024Neural Visibility Field for Uncertainty-Driven Active Mappingcited by 25
- 2021A Survey on Human-aware Robot Navigationcited by 23
- 2022EgoEnv: Human-centric environment representations from egocentric videocited by 22
- 2022PEANUT: Predicting and Navigating to Unseen Targetscited by 22
- 2023Measuring Acoustics with Collaborative Multiple Agentscited by 21
- 2021GridToPix: Training Embodied Agents with Minimal Supervisioncited by 20
- 2022Learning Active Camera for Multi-Object Navigationcited by 20
- 2021Learning Efficient Multi-Agent Cooperative Visual Explorationcited by 19
- 2025ActiveGAMER: Active GAussian Mapping through Efficient Renderingcited by 19
- 2022Ask4Help: Learning to Leverage an Expert for Embodied Taskscited by 18
- 2022Pay Self-Attention to Audio-Visual Navigationcited by 17
- 2024OVER-NAV: Elevating Iterative Vision-and-Language Navigation with Open-Vocabulary Detection and StructurEd Representationcited by 17
- 2025ORV: 4D Occupancy-centric Robot Video Generationcited by 16
- 2022Navigating to Objects in Unseen Environments by Distance Predictioncited by 15
- 2025AutoOcc: Automatic Open-Ended Semantic Occupancy Annotation via Vision-Language Guided Gaussian Splattingcited by 14
- 2023A Domain-Agnostic Approach for Characterization of Lifelong Learning Systemscited by 13
- 2025NextBestPath: Efficient 3D Mapping of Unseen Environmentscited by 13
- 2021Environment Predictive Coding for Embodied Agentscited by 12
- 2021MaAST: Map Attention with Semantic Transformersfor Efficient Visual Navigationcited by 11
- 2023MOPA: Modular Object Navigation with PointGoal Agentscited by 11
- 2025GLEAM: Learning Generalizable Exploration Policy for Active Mapping in Complex 3D Indoor Scenescited by 10
- 2024MapEx: Indoor Structure Exploration with Probabilistic Information Gain from Global Map Predictionscited by 9
- 2021Out of the Box: Embodied Navigation in the Real Worldcited by 8
- 2021Focus on Impact: Indoor Exploration with Intrinsic Motivationcited by 8
- 2022Multi-Robot Active Mapping via Neural Bipartite Graph Matchingcited by 8
- 2022What do navigation agents learn about their environment?cited by 8
50 most cited of 104 All 104 (JSON)
- 2026Rethinking Learned Occupancy in Autonomous Active Mapping with Observation-Gated Filtering8 Sep 2026
- 2026Diagnosing and Dynamically Filtering Occupancy World Models for Active Mapping6 Sep 2026
- 2026NavWAM: A Navigation World Action Model for Goal-Conditioned Visual Navigation11 Jun 2026
- 20263D Gaussian Map with Open-Set Semantic Grouping for Vision-Language Navigation26 May 2026
- 2026Remember to be Curious: Episodic Context and Persistent Worlds for 3D Exploration21 May 2026
- 2026MAGICIAN: Efficient Long-Term Planning with Imagined Gaussians for Active Mapping23 Mar 2026
- 2026FlatLands: Generative Floormap Completion From a Single Egocentric View16 Mar 2026
- 2026How Far Can We Go with Pixels Alone? A Pilot Study on Screen-Only Navigation in Commercial 3D ARPGs21 Feb 2026
- 2025Schr"odinger's Navigator: Imagining an Ensemble of Futures for Zero-Shot Object Navigation24 Dec 2025
- 2025ImagineNav++: Prompting Vision-Language Models as Embodied Navigator through Scene Imagination19 Dec 2025
- 2025Scaling Up Occupancy-centric Driving Scene Generation: Dataset and Method27 Oct 2025
- 2025Dream to Recall: Imagination-Guided Experience Retrieval for Memory-Persistent Vision-and-Language Navigation9 Oct 2025
38 more
- 2025Efficient Navigation in Unknown Indoor Environments with Vision-Language Models6 Oct 2025
- 2025Collaborative Perceiver: Elevating Vision-based 3D Object Detection via Local Density-Aware Spatial Occupancy28 Jul 2025
- 2025Peering into the Unknown: Active View Selection with Neural Uncertainty Maps for 3D Reconstruction17 Jun 2025
- 2025ORV: 4D Occupancy-centric Robot Video Generation3 Jun 2025
- 2025Understanding while Exploring: Semantics-driven Active Mapping30 May 2025
- 2025GLEAM: Learning Generalizable Exploration Policy for Active Mapping in Complex 3D Indoor Scenes26 May 2025
- 2025TACOcc:Target-Adaptive Cross-Modal Fusion with Volume Rendering for 3D Semantic Occupancy19 May 2025
- 2025NextBestPath: Efficient 3D Mapping of Unseen Environments7 Feb 2025
- 2025AutoOcc: Automatic Open-Ended Semantic Occupancy Annotation via Vision-Language Guided Gaussian Splatting7 Feb 2025
- 2025VL-Nav: Neuro-Symbolic Reasoning-based Vision-Language Navigation2 Feb 2025
- 2025ActiveGAMER: Active GAussian Mapping through Efficient Rendering12 Jan 2025
- 2024Active Neural Mapping at Scale30 Sep 2024
- 2024MapEx: Indoor Structure Exploration with Probabilistic Information Gain from Global Map Predictions23 Sep 2024
- 2024Neural Visibility Field for Uncertainty-Driven Active Mapping11 Jun 2024
- 2024MAGIC: Map-Guided Few-Shot Audio-Visual Acoustics Modeling22 May 2024
- 2024HELPER-X: A Unified Instructable Embodied Agent to Tackle Four Interactive Vision-Language Domains with Memory-Augmented Language Models29 Apr 2024
- 2024GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigation9 Apr 2024
- 2024OVER-NAV: Elevating Iterative Vision-and-Language Navigation with Open-Vocabulary Detection and StructurEd Representation26 Mar 2024
- 2024Leveraging Large Language Model-based Room-Object Relationships Knowledge for Enhancing Multimodal-Input Object Goal Navigation21 Mar 2024
- 2024Mapping High-level Semantic Regions in Indoor Environments without Object Recognition11 Mar 2024
- 2024NARUTO: Neural Active Reconstruction from Uncertain Target Observations29 Feb 2024
- 20244CNet: A Diffusion Approach to Map Prediction for Decentralized Multi-Robot Exploration27 Feb 2024
- 2024Opening Articulated Structures in the Real World27 Feb 2024
- 2023Invariant-based Mapping of Space During General Motion of an Observer18 Nov 2023
- 2023Measuring Acoustics with Collaborative Multiple Agents9 Oct 2023
- 2023DREAM: Decentralized Reinforcement Learning for Exploration and Efficient Energy Management in Multi-Robot Systems29 Sep 2023
- 2023What Is Near?: Room Locality Learning for Enhanced Robot Vision-Language-Navigation in Indoor Living Environments10 Sep 2023
- 2023Active Neural Mapping30 Aug 2023
- 2023Calibrating Panoramic Depth Estimation for Practical Localization and Mapping27 Aug 2023
- 2023Sonicverse: A Multisensory Simulation Platform for Embodied Household Agents that See and Hear1 Jun 2023
- 2023MOPA: Modular Object Navigation with PointGoal Agents7 Apr 2023
- 2023SePaint: Semantic Map Inpainting via Multinomial Diffusion5 Mar 2023
- 2023Renderable Neural Radiance Map for Visual Navigation1 Mar 2023
- 2023A Domain-Agnostic Approach for Characterization of Lifelong Learning Systems18 Jan 2023
- 2023Embodied Agents for Efficient Exploration and Smart Scene Description17 Jan 2023
- 2023ReVoLT: Relational Reasoning and Voronoi Local Graph Planning for Target-driven Navigation6 Jan 2023
- 2023Chat2Map: Efficient Scene Mapping from Multi-Ego Conversations4 Jan 2023
- 2022Comparison of Model-Free and Model-Based Learning-Informed Planning for PointGoal Navigation17 Dec 2022
50 newest of 104 All 104 (JSON)
- 2024OVER-NAV: Elevating Iterative Vision-and-Language Navigation with Open-Vocabulary Detection and StructurEd Representationcited by 17
- 2025ORV: 4D Occupancy-centric Robot Video Generationcited by 16
- 2026NavWAM: A Navigation World Action Model for Goal-Conditioned Visual Navigationcited by 7
- 2022Tracking and Planning with Spatial World Modelscited by 3
- 2025Dream to Recall: Imagination-Guided Experience Retrieval for Memory-Persistent Vision-and-Language Navigationcited by 3
- 2026Diagnosing and Dynamically Filtering Occupancy World Models for Active Mappingcited by 0
- 2026Diagnosing and Dynamically Filtering Occupancy World Models for Active Mapping6 Sep 2026
- 2026NavWAM: A Navigation World Action Model for Goal-Conditioned Visual Navigation11 Jun 2026
- 2025Dream to Recall: Imagination-Guided Experience Retrieval for Memory-Persistent Vision-and-Language Navigation9 Oct 2025
- 2025ORV: 4D Occupancy-centric Robot Video Generation3 Jun 2025
- 2024OVER-NAV: Elevating Iterative Vision-and-Language Navigation with Open-Vocabulary Detection and StructurEd Representation26 Mar 2024
- 2022Tracking and Planning with Spatial World Models25 Jan 2022
- 2025AutoOcc: Automatic Open-Ended Semantic Occupancy Annotation via Vision-Language Guided Gaussian Splattingcited by 14
- 2025Efficient Navigation in Unknown Indoor Environments with Vision-Language Modelscited by 2
- 2025ImagineNav++: Prompting Vision-Language Models as Embodied Navigator through Scene Imaginationcited by 1
- 2025ImagineNav++: Prompting Vision-Language Models as Embodied Navigator through Scene Imagination19 Dec 2025
- 2025Efficient Navigation in Unknown Indoor Environments with Vision-Language Models6 Oct 2025
- 2025AutoOcc: Automatic Open-Ended Semantic Occupancy Annotation via Vision-Language Guided Gaussian Splatting7 Feb 2025
- 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
- 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 501
- 2021A Survey of Embodied AI: From Simulators to Research Taskscited by 256
- 2022PONI: Potential Functions for ObjectGoal Navigation with Interaction-free Learningcited by 149
- 2022Navigating to Objects in the Real Worldcited by 104
- 2021Visual Room Rearrangementcited by 101
- 2024GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigationcited by 93
- 2022Cross-modal Map Learning for Vision and Language Navigationcited by 91
- 2022Weakly-Supervised Multi-Granularity Map Learning for Vision-and-Language Navigationcited by 86
- 2022Zero Experience Required: Plug & Play Modular Transfer Learning for Semantic Visual Navigationcited by 66
- 2021The ThreeDWorld Transport Challenge: A Visually Guided Task-and-Motion Planning Benchmark for Physically Realistic Embodied AIcited by 64
- 2021No RL, No Simulation: Learning to Navigate without Navigatingcited by 63
12 most cited of 92 All 92 (JSON)
- 2026Rethinking Learned Occupancy in Autonomous Active Mapping with Observation-Gated Filtering8 Sep 2026
- 20263D Gaussian Map with Open-Set Semantic Grouping for Vision-Language Navigation26 May 2026
- 2026Remember to be Curious: Episodic Context and Persistent Worlds for 3D Exploration21 May 2026
- 2026MAGICIAN: Efficient Long-Term Planning with Imagined Gaussians for Active Mapping23 Mar 2026
- 2026FlatLands: Generative Floormap Completion From a Single Egocentric View16 Mar 2026
- 2026How Far Can We Go with Pixels Alone? A Pilot Study on Screen-Only Navigation in Commercial 3D ARPGs21 Feb 2026
- 2025Schr"odinger's Navigator: Imagining an Ensemble of Futures for Zero-Shot Object Navigation24 Dec 2025
- 2025Scaling Up Occupancy-centric Driving Scene Generation: Dataset and Method27 Oct 2025
- 2025Collaborative Perceiver: Elevating Vision-based 3D Object Detection via Local Density-Aware Spatial Occupancy28 Jul 2025
- 2025Peering into the Unknown: Active View Selection with Neural Uncertainty Maps for 3D Reconstruction17 Jun 2025
- 2025Understanding while Exploring: Semantics-driven Active Mapping30 May 2025
- 2025GLEAM: Learning Generalizable Exploration Policy for Active Mapping in Complex 3D Indoor Scenes26 May 2025
12 newest of 92 All 92 (JSON)