# Seeing the Un-Scene: Learning Amodal Semantic Maps for Room Navigation

arXiv: https://arxiv.org/abs/2007.09841 · HTML: https://mlsanity.com/paper/2007.09841/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jul 2020.

## Built on by (most cited first, 30 of 34)

- [A Survey of Embodied AI: From Simulators to Research Tasks](https://mlsanity.com/paper/2103.04918/) (2021)
- [Diffusion-based Generation, Optimization, and Planning in 3D Scenes](https://mlsanity.com/paper/2301.06015/) (2023)
- [BEVBert: Multimodal Map Pre-training for Language-guided Navigation](https://mlsanity.com/paper/2212.04385/) (2022)
- [Pathdreamer: A World Model for Indoor Navigation](https://mlsanity.com/paper/2105.08756/) (2021)
- [Cross-modal Map Learning for Vision and Language Navigation](https://mlsanity.com/paper/2203.05137/) (2022)
- [Zero Experience Required: Plug & Play Modular Transfer Learning for Semantic Visual Navigation](https://mlsanity.com/paper/2202.02440/) (2022)
- [ViKiNG: Vision-Based Kilometer-Scale Navigation with Geographic Hints](https://mlsanity.com/paper/2202.11271/) (2022)
- [Housekeep: Tidying Virtual Households using Commonsense Reasoning](https://mlsanity.com/paper/2205.10712/) (2022)
- [Learning to Map for Active Semantic Goal Navigation](https://mlsanity.com/paper/2106.15648/) (2021)
- [SASRA: Semantically-aware Spatio-temporal Reasoning Agent for Vision-and-Language Navigation in Continuous Environments](https://mlsanity.com/paper/2108.11945/) (2021)
- [Unseen Object Amodal Instance Segmentation via Hierarchical Occlusion Modeling](https://mlsanity.com/paper/2109.11103/) (2021)
- [Uncertainty-driven Planner for Exploration and Navigation](https://mlsanity.com/paper/2202.11907/) (2022)
- [The Surprising Effectiveness of Visual Odometry Techniques for Embodied PointGoal Navigation](https://mlsanity.com/paper/2108.11550/) (2021)
- [PEANUT: Predicting and Navigating to Unseen Targets](https://mlsanity.com/paper/2212.02497/) (2022)
- [Image Amodal Completion: A Survey](https://mlsanity.com/paper/2207.02062/) (2022)
- [Object-level Scene Deocclusion](https://mlsanity.com/paper/2406.07706/) (2024)
- [Navigating to Objects in Unseen Environments by Distance Prediction](https://mlsanity.com/paper/2202.03735/) (2022)
- [Environment Predictive Coding for Embodied Agents](https://mlsanity.com/paper/2102.02337/) (2021)
- [Chat2Map: Efficient Scene Mapping from Multi-Ego Conversations](https://mlsanity.com/paper/2301.02184/) (2023)
- [WAFFLE: Multimodal Floorplan Understanding in the Wild](https://mlsanity.com/paper/2412.00955/) (2024)
- [Probable Object Location (POLo) Score Estimation for Efficient Object Goal Navigation](https://mlsanity.com/paper/2311.07992/) (2023)
- [3D Gaussian Map with Open-Set Semantic Grouping for Vision-Language Navigation](https://mlsanity.com/paper/2605.26500/) (2026)
- [Deep Learning for Embodied Vision Navigation: A Survey](https://mlsanity.com/paper/2108.04097/) (2021)
- [Image-based Navigation in Real-World Environments via Multiple Mid-level Representations: Fusion Models, Benchmark and Efficient Evaluation](https://mlsanity.com/paper/2202.01069/) (2022)
- [Boosting Semantic Segmentation with Semantic Boundaries](https://mlsanity.com/paper/2304.09427/) (2023)
- [What Is Near?: Room Locality Learning for Enhanced Robot Vision-Language-Navigation in Indoor Living Environments](https://mlsanity.com/paper/2309.05036/) (2023)
- [Mapping High-level Semantic Regions in Indoor Environments without Object Recognition](https://mlsanity.com/paper/2403.07076/) (2024)
- [Cross-Dimensional Refined Learning for Real-Time 3D Visual Perception from Monocular Video](https://mlsanity.com/paper/2303.09248/) (2023)
- [Raster2Seq: Polygon Sequence Generation for Floorplan Reconstruction](https://mlsanity.com/paper/2602.09016/) (2026)
- [Realistic PointGoal Navigation via Auxiliary Losses and Information Bottleneck](https://mlsanity.com/paper/2109.08677/) (2021)

## Abstract (arXiv)

We introduce a learning-based approach for room navigation using semantic maps. Our proposed architecture learns to predict top-down belief maps of regions that lie beyond the agent's field of view while modeling architectural and stylistic regularities in houses. First, we train a model to generate amodal semantic top-down maps indicating beliefs of location, size, and shape of rooms by learning the underlying architectural patterns in houses. Next, we use these maps to predict a point that lies in the target room and train a policy to navigate to the point. We empirically demonstrate that by predicting semantic maps, the model learns common correlations found in houses and generalizes to novel environments. We also demonstrate that reducing the task of room navigation to point navigation improves the performance further.