# 3D VSG: Long-term Semantic Scene Change Prediction through 3D Variable Scene Graphs

arXiv: https://arxiv.org/abs/2209.07896 · HTML: https://mlsanity.com/paper/2209.07896/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 56 references that are in the map.
- First posted on arXiv 16 Sep 2022.

## Builds on (most cited first, 30 of 34)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classification](https://mlsanity.com/paper/2009.03509/) (2020)
- [Visual Relationship Detection with Language Priors](https://mlsanity.com/paper/1608.00187/) (2016)
- [Fully Convolutional Siamese Networks for Change Detection](https://mlsanity.com/paper/1810.08462/) (2018)
- [Graph R-CNN for Scene Graph Generation](https://mlsanity.com/paper/1808.00191/) (2018)
- [3D Scene Graph: A Structure for Unified Semantics, 3D Space, and Camera](https://mlsanity.com/paper/1910.02527/) (2019)
- [Learning 3D Semantic Scene Graphs from 3D Indoor Reconstructions](https://mlsanity.com/paper/2004.03967/) (2020)
- [RIO: 3D Object Instance Re-Localization in Changing Indoor Environments](https://mlsanity.com/paper/1908.06109/) (2019)
- [Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mapping](https://mlsanity.com/paper/1910.02490/) (2019)
- [OccuSeg: Occupancy-aware 3D Instance Segmentation](https://mlsanity.com/paper/2003.06537/) (2020)
- [GPS-Net: Graph Property Sensing Network for Scene Graph Generation](https://mlsanity.com/paper/2003.12962/) (2020)
- [Populating 3D Scenes by Learning Human-Scene Interaction](https://mlsanity.com/paper/2012.11581/) (2020)
- [SceneGraphFusion: Incremental 3D Scene Graph Prediction from RGB-D Sequences](https://mlsanity.com/paper/2103.14898/) (2021)
- [3D Dynamic Scene Graphs: Actionable Spatial Perception with Places, Objects, and Humans](https://mlsanity.com/paper/2002.06289/) (2020)
- [SemanticFusion: Dense 3D Semantic Mapping with Convolutional Neural Networks](https://mlsanity.com/paper/1609.05130/) (2016)
- [Visual Semantic Navigation using Scene Priors](https://mlsanity.com/paper/1810.06543/) (2018)
- [LayoutTransformer: Layout Generation and Completion with Self-attention](https://mlsanity.com/paper/2006.14615/) (2020)
- [Spatially-Aware Graph Neural Networks for Relational Behavior Forecasting from Sensor Data](https://mlsanity.com/paper/1910.08233/) (2019)
- [Fusion++: Volumetric Object-Level SLAM](https://mlsanity.com/paper/1808.08378/) (2018)
- [Graph-to-3D: End-to-End Generation and Manipulation of 3D Scenes Using Scene Graphs](https://mlsanity.com/paper/2108.08841/) (2021)
- [Hierarchical Planning for Long-Horizon Manipulation with Geometric and Symbolic Scene Graphs](https://mlsanity.com/paper/2012.07277/) (2020)
- [Volumetric Instance-Aware Semantic Mapping and 3D Object Discovery](https://mlsanity.com/paper/1903.00268/) (2019)
- [TASKOGRAPHY: Evaluating robot task planning over large 3D scene graphs](https://mlsanity.com/paper/2207.05006/) (2022)
- [Indoor Semantic Segmentation using depth information](https://mlsanity.com/paper/1301.3572/) (2013)
- [End-to-End Optimization of Scene Layout](https://mlsanity.com/paper/2007.11744/) (2020)
- [Hierarchical Representations and Explicit Memory: Learning Effective Navigation Policies on 3D Scene Graphs using Graph Neural Networks](https://mlsanity.com/paper/2108.01176/) (2021)
- [Panoptic Multi-TSDFs: a Flexible Representation for Online Multi-resolution Volumetric Mapping and Long-term Dynamic Scene Consistency](https://mlsanity.com/paper/2109.10165/) (2021)
- [ChangeSim: Towards End-to-End Online Scene Change Detection in Industrial Indoor Environments](https://mlsanity.com/paper/2103.05368/) (2021)
- [Retargetable AR: Context-aware Augmented Reality in Indoor Scenes based on 3D Scene Graph](https://mlsanity.com/paper/2008.07817/) (2020)

## Built on by (most cited first, 24 of 24)

- [Open3DSG: Open-Vocabulary 3D Scene Graphs from Point Clouds with Queryable Objects and Open-Set Relationships](https://mlsanity.com/paper/2402.12259/) (2024)
- [When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Models](https://mlsanity.com/paper/2405.10255/) (2024)
- [SGAligner : 3D Scene Alignment with Scene Graphs](https://mlsanity.com/paper/2304.14880/) (2023)
- [SceneGraphLoc: Cross-Modal Coarse Visual Localization on 3D Scene Graphs](https://mlsanity.com/paper/2404.00469/) (2024)
- [Lang3DSG: Language-based contrastive pre-training for 3D Scene Graph prediction](https://mlsanity.com/paper/2310.16494/) (2023)
- [Describe Anything Anywhere At Any Moment](https://mlsanity.com/paper/2512.00565/) (2025)
- [RelationField: Relate Anything in Radiance Fields](https://mlsanity.com/paper/2412.13652/) (2024)
- [Lost & Found: Tracking Changes from Egocentric Observations in 3D Dynamic Scene Graphs](https://mlsanity.com/paper/2411.19162/) (2024)
- [Aion: Towards Hierarchical 4D Scene Graphs with Temporal Flow Dynamics](https://mlsanity.com/paper/2512.11903/) (2025)
- [Belief Scene Graphs: Expanding Partial Scenes with Objects through Computation of Expectation](https://mlsanity.com/paper/2402.03840/) (2024)
- [SceneDiff: A Benchmark and Method for Multiview Object Change Detection](https://mlsanity.com/paper/2512.16908/) (2025)
- [3D Scene Graphs: Open Challenges and Future Directions](https://mlsanity.com/paper/2606.19383/) (2026)
- [Estimating Commonsense Scene Composition on Belief Scene Graphs](https://mlsanity.com/paper/2505.02405/) (2025)
- [From Pixels to Primitives: Scene Change Detection in 3D Gaussian Splatting](https://mlsanity.com/paper/2605.07203/) (2026)
- [BYE: Build Your Encoder with One Sequence of Exploration Data for Long-Term Dynamic Scene Understanding](https://mlsanity.com/paper/2412.02449/) (2024)
- [Perpetua: Multi-Hypothesis Persistence Modeling for Semi-Static Environments](https://mlsanity.com/paper/2507.18808/) (2025)
- [Integrating Prior Observations for Incremental 3D Scene Graph Prediction](https://mlsanity.com/paper/2509.11895/) (2025)
- [M3DMap: Object-aware Multimodal 3D Mapping for Dynamic Environments](https://mlsanity.com/paper/2508.17044/) (2025)
- [ChangingGrounding: 3D Visual Grounding in Changing Scenes](https://mlsanity.com/paper/2510.14965/) (2025)
- [VIZOR: Viewpoint-Invariant Zero-Shot Scene Graph Generation for 3D Scene Reasoning](https://mlsanity.com/paper/2602.00637/) (2026)
- [MA3DSG: Multi-Agent 3D Scene Graph Generation for Large-Scale Indoor Environments](https://mlsanity.com/paper/2602.04152/) (2026)
- [TERDNet: Transformer Encoder-Recurrent Decoder Network for Scene Change Detection](https://mlsanity.com/paper/2605.20822/) (2026)
- [Prior-SG: Task and Prior Driven Region Segmentation for Scene Graphs in Arbitrarily-Structured Environments](https://mlsanity.com/paper/2608.06170/) (2026)
- [Kairos: Grounded Forecasting of Presence and Directional Flow in 4D Scene Graphs](https://mlsanity.com/paper/2609.27467/) (2026)

## Abstract (arXiv)

Numerous applications require robots to operate in environments shared with other agents, such as humans or other robots. However, such shared scenes are typically subject to different kinds of long-term semantic scene changes. The ability to model and predict such changes is thus crucial for robot autonomy. In this work, we formalize the task of semantic scene variability estimation and identify three main varieties of semantic scene change: changes in the position of an object, its semantic state, or the composition of a scene as a whole. To represent this variability, we propose the Variable Scene Graph (VSG), which augments existing 3D Scene Graph (SG) representations with the variability attribute, representing the likelihood of discrete long-term change events. We present a novel method, DeltaVSG, to estimate the variability of VSGs in a supervised fashion. We evaluate our method on the 3RScan long-term dataset, showing notable improvements in this novel task over existing approaches. Our method DeltaVSG achieves an accuracy of 77.1% and a recall of 72.3%, often mimicking human intuition about how indoor scenes change over time. We further show the utility of VSG prediction in the task of active robotic change detection, speeding up task completion by 66.0% compared to a scene-change-unaware planner. We make our code available as open-source.