# Belief Scene Graphs: Expanding Partial Scenes with Objects through Computation of Expectation

arXiv: https://arxiv.org/abs/2402.03840 · HTML: https://mlsanity.com/paper/2402.03840/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 27 references that are in the map.
- First posted on arXiv 6 Feb 2024.

## Builds on (most cited first, 22 of 22)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AI](https://mlsanity.com/paper/2109.08238/) (2021)
- [3D Scene Graph: A Structure for Unified Semantics, 3D Space, and Camera](https://mlsanity.com/paper/1910.02527/) (2019)
- [Learning 3D Semantic Scene Graphs from 3D Indoor Reconstructions](https://mlsanity.com/paper/2004.03967/) (2020)
- [Hydra: A Real-time Spatial Perception System for 3D Scene Graph Construction and Optimization](https://mlsanity.com/paper/2201.13360/) (2022)
- [On Wasserstein Two Sample Testing and Related Families of Nonparametric Tests](https://mlsanity.com/paper/1509.02237/) (2015)
- [SceneGraphFusion: Incremental 3D Scene Graph Prediction from RGB-D Sequences](https://mlsanity.com/paper/2103.14898/) (2021)
- [Kimera: from SLAM to Spatial Perception with 3D Dynamic Scene Graphs](https://mlsanity.com/paper/2101.06894/) (2021)
- [A Comprehensive Survey of Scene Graphs: Generation and Application](https://mlsanity.com/paper/2104.01111/) (2021)
- [3-D Scene Graph: A Sparse and Semantic Representation of Physical Environments for Intelligent Agents](https://mlsanity.com/paper/1908.04929/) (2019)
- [Hierarchical Representations and Explicit Memory: Learning Effective Navigation Policies on 3D Scene Graphs using Graph Neural Networks](https://mlsanity.com/paper/2108.01176/) (2021)
- [VL-SAT: Visual-Linguistic Semantics Assisted Training for 3D Semantic Scene Graph Prediction in Point Cloud](https://mlsanity.com/paper/2303.14408/) (2023)
- [Incremental 3D Semantic Scene Graph Prediction from RGB Sequences](https://mlsanity.com/paper/2305.02743/) (2023)
- [Reasoning with Scene Graphs for Robot Planning under Partial Observability](https://mlsanity.com/paper/2202.10432/) (2022)
- [3D VSG: Long-term Semantic Scene Change Prediction through 3D Variable Scene Graphs](https://mlsanity.com/paper/2209.07896/) (2022)
- [Situational Graphs for Robot Navigation in Structured Indoor Environments](https://mlsanity.com/paper/2202.12197/) (2022)
- [Semantic and Geometric Modeling with Neural Message Passing in 3D Scene Graphs for Hierarchical Mechanical Search](https://mlsanity.com/paper/2012.04060/) (2020)
- [Towards Traffic Scene Description: The Semantic Scene Graph](https://mlsanity.com/paper/2111.10196/) (2021)
- [Spatial Commonsense Graph for Object Localisation in Partial Scenes](https://mlsanity.com/paper/2203.05380/) (2022)
- [Safe and Efficient Navigation in Extreme Environments using Semantic Belief Graphs](https://mlsanity.com/paper/2304.00645/) (2023)

## Built on by (most cited first, 4 of 4)

- [3D Scene Graphs: Open Challenges and Future Directions](https://mlsanity.com/paper/2606.19383/) (2026)
- [Estimating Commonsense Scene Composition on Belief Scene Graphs](https://mlsanity.com/paper/2505.02405/) (2025)
- [FOGMACHINE -- Leveraging Discrete-Event Simulation and Scene Graphs for Modeling Hierarchical, Interconnected Environments under Partial Observations from Mobile Agents](https://mlsanity.com/paper/2510.09483/) (2025)
- [Hierarchical Aggregation of Semantic Uncertainty in 3D Scene Graphs](https://mlsanity.com/paper/2609.22351/) (2026)

## Abstract (arXiv)

In this article, we propose the novel concept of Belief Scene Graphs, which are utility-driven extensions of partial 3D scene graphs, that enable efficient high-level task planning with partial information. We propose a graph-based learning methodology for the computation of belief (also referred to as expectation) on any given 3D scene graph, which is then used to strategically add new nodes (referred to as blind nodes) that are relevant to a robotic mission. We propose the method of Computation of Expectation based on Correlation Information (CECI), to reasonably approximate real Belief/Expectation, by learning histograms from available training data. A novel Graph Convolutional Neural Network (GCN) model is developed, to learn CECI from a repository of 3D scene graphs. As no database of 3D scene graphs exists for the training of the novel CECI model, we present a novel methodology for generating a 3D scene graph dataset based on semantically annotated real-life 3D spaces. The generated dataset is then utilized to train the proposed CECI model and for extensive validation of the proposed method. We establish the novel concept of textitBelief Scene Graphs (BSG), as a core component to integrate expectations into abstract representations. This new concept is an evolution of the classical 3D scene graph concept and aims to enable high-level reasoning for task planning and optimization of a variety of robotics missions. The efficacy of the overall framework has been evaluated in an object search scenario, and has also been tested in a real-life experiment to emulate human common sense of unseen-objects. For a video of the article, showcasing the experimental demonstration, please refer to the following link: https://youtu.be/hsGlSCa12iY