# Indoor Scene Understanding in 2.5/3D for Autonomous Agents: A Survey

arXiv: https://arxiv.org/abs/1803.03352 · HTML: https://mlsanity.com/paper/1803.03352/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Mar 2018.

## Built on by (most cited first, 25 of 25)

- [Open-Fusion: Real-time Open-Vocabulary 3D Mapping and Queryable Scene Representation](https://mlsanity.com/paper/2310.03923/) (2023)
- [Multi-Modal Data-Efficient 3D Scene Understanding for Autonomous Driving](https://mlsanity.com/paper/2405.05258/) (2024)
- [Deep Learning Based 3D Segmentation: A Survey](https://mlsanity.com/paper/2103.05423/) (2021)
- [Agent3D-Zero: An Agent for Zero-shot 3D Understanding](https://mlsanity.com/paper/2403.11835/) (2024)
- [Contrastive Gaussian Clustering: Weakly Supervised 3D Scene Segmentation](https://mlsanity.com/paper/2404.12784/) (2024)
- [FreDSNet: Joint Monocular Depth and Semantic Segmentation with Fast Fourier Convolutions](https://mlsanity.com/paper/2210.01595/) (2022)
- [Beyond RGB: Scene-Property Synthesis with Neural Radiance Fields](https://mlsanity.com/paper/2206.04669/) (2022)
- [Multi-view 3D Object Reconstruction and Uncertainty Modelling with Neural Shape Prior](https://mlsanity.com/paper/2306.11739/) (2023)
- [FunRec: Reconstructing Functional 3D Scenes from Egocentric Interaction Videos](https://mlsanity.com/paper/2604.05621/) (2026)
- [Toward General Object-level Mapping from Sparse Views with 3D Diffusion Priors](https://mlsanity.com/paper/2410.05514/) (2024)
- [Do You See What I See? Capabilities and Limits of Automated Multimedia Content Analysis](https://mlsanity.com/paper/2201.11105/) (2021)
- [Atlanta Scaled layouts from non-central panoramas](https://mlsanity.com/paper/2401.17058/) (2024)
- [Rethinking Data Input for Point Cloud Upsampling](https://mlsanity.com/paper/2407.04476/) (2024)
- [Learning to Assess Danger from Movies for Cooperative Escape Planning in Hazardous Environments](https://mlsanity.com/paper/2207.13791/) (2022)
- [Advancing Complex Wide-Area Scene Understanding with Hierarchical Coresets Selection](https://mlsanity.com/paper/2507.13061/) (2025)
- [LLaVA3: Representing 3D Scenes like a Cubist Painter to Boost 3D Scene Understanding of VLMs](https://mlsanity.com/paper/2511.16454/) (2025)
- [Grab-3D: Detecting AI-Generated Videos from 3D Geometric Temporal Consistency](https://mlsanity.com/paper/2512.13665/) (2025)
- [A Dynamic Data Driven Approach for Explainable Scene Understanding](https://mlsanity.com/paper/2206.09089/) (2022)
- [Background Invariance Testing According to Semantic Proximity](https://mlsanity.com/paper/2208.09286/) (2022)
- [Review on 6D Object Pose Estimation with the focus on Indoor Scene Understanding](https://mlsanity.com/paper/2212.01920/) (2022)
- [Automatic Spatial Calibration of Near-Field MIMO Radar With Respect to Optical Depth Sensors](https://mlsanity.com/paper/2403.10981/) (2024)
- [A Scalable Quantum Non-local Neural Network for Image Classification](https://mlsanity.com/paper/2407.18906/) (2024)
- [Leveraging Foundation Models To learn the shape of semi-fluid deformable objects](https://mlsanity.com/paper/2411.16802/) (2024)
- [Break Out the Silverware -- Semantic Understanding of Stored Household Items](https://mlsanity.com/paper/2512.23739/) (2025)
- [Semantic-ITC: A Frame-wise Indoor Mobile Laser Scanning Dataset and Benchmark for Semantic Segmentation](https://mlsanity.com/paper/2609.18493/) (2026)

## Abstract (arXiv)

With the availability of low-cost and compact 2.5/3D visual sensing devices, computer vision community is experiencing a growing interest in visual scene understanding of indoor environments. This survey paper provides a comprehensive background to this research topic. We begin with a historical perspective, followed by popular 3D data representations and a comparative analysis of available datasets. Before delving into the application specific details, this survey provides a succinct introduction to the core technologies that are the underlying methods extensively used in the literature. Afterwards, we review the developed techniques according to a taxonomy based on the scene understanding tasks. This covers holistic indoor scene understanding as well as subtasks such as scene classification, object detection, pose estimation, semantic segmentation, 3D reconstruction, saliency detection, physics-based reasoning and affordance prediction. Later on, we summarize the performance metrics used for evaluation in different tasks and a quantitative comparison among the recent state-of-the-art techniques. We conclude this review with the current challenges and an outlook towards the open research problems requiring further investigation.