# Online Learning of a Probabilistic and Adaptive Scene Representation

arXiv: https://arxiv.org/abs/2103.16832 · HTML: https://mlsanity.com/paper/2103.16832/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 73 references that are in the map.
- First posted on arXiv 31 Mar 2021.

## Builds on (most cited first, 22 of 22)

- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations](https://mlsanity.com/paper/1906.01618/) (2019)
- [Convolutional Occupancy Networks](https://mlsanity.com/paper/2003.04618/) (2020)
- [Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision](https://mlsanity.com/paper/1912.07372/) (2019)
- [Local Implicit Grid Representations for 3D Scenes](https://mlsanity.com/paper/2003.08981/) (2020)
- [Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstruction](https://mlsanity.com/paper/2003.10983/) (2020)
- [A Neural Dirichlet Process Mixture Model for Task-Free Continual Learning](https://mlsanity.com/paper/2001.00689/) (2020)
- [BundleFusion: Real-time Globally Consistent 3D Reconstruction using On-the-fly Surface Re-integration](https://mlsanity.com/paper/1604.01093/) (2016)
- [RoutedFusion: Learning Real-time Depth Map Fusion](https://mlsanity.com/paper/2001.04388/) (2020)
- [FilterReg: Robust and Efficient Probabilistic Point-Set Registration using Gaussian Filter and Twist Parameterization](https://mlsanity.com/paper/1811.10136/) (2018)
- [FutureMapping: The Computational Structure of Spatial AI Systems](https://mlsanity.com/paper/1803.11288/) (2018)
- [PointGMM: a Neural GMM Network for Point Clouds](https://mlsanity.com/paper/2003.13326/) (2020)
- [SurfelMeshing: Online Surfel-Based Mesh Reconstruction](https://mlsanity.com/paper/1810.00729/) (2018)
- [Joint Alignment of Multiple Point Sets with Batch and Incremental Expectation-Maximization](https://mlsanity.com/paper/1609.01466/) (2016)
- [PSDF Fusion: Probabilistic Signed Distance Function for On-the-fly 3D Data Fusion and Scene Reconstruction](https://mlsanity.com/paper/1807.11034/) (2018)
- [An Efficient Volumetric Mesh Representation for Real-time Scene Reconstruction using Spatial Hashing](https://mlsanity.com/paper/1803.03949/) (2018)
- [Streaming, Distributed Variational Inference for Bayesian Nonparametrics](https://mlsanity.com/paper/1510.09161/) (2015)
- [MRFMap: Online Probabilistic 3D Mapping using Forward Ray Sensor Models](https://mlsanity.com/paper/2006.03512/) (2020)
- [Warped Gaussian Processes Occupancy Mapping with Uncertain Inputs](https://mlsanity.com/paper/1701.00925/) (2017)
- [Direct Fitting of Gaussian Mixture Models](https://mlsanity.com/paper/1904.05537/) (2019)
- [Cubic Range Error Model for Stereo Vision with Illuminators](https://mlsanity.com/paper/1803.03932/) (2018)

## Built on by (most cited first, 1 of 1)

- [Probabilistic Point Cloud Modeling via Self-Organizing Gaussian Mixture Models](https://mlsanity.com/paper/2302.00047/) (2023)

## Abstract (arXiv)

Constructing and maintaining a consistent scene model on-the-fly is the core task for online spatial perception, interpretation, and action. In this paper, we represent the scene with a Bayesian nonparametric mixture model, seamlessly describing per-point occupancy status with a continuous probability density function. Instead of following the conventional data fusion paradigm, we address the problem of online learning the process how sequential point cloud data are generated from the scene geometry. An incremental and parallel inference is performed to update the parameter space in real-time. We experimentally show that the proposed representation achieves state-of-the-art accuracy with promising efficiency. The consistent probabilistic formulation assures a generative model that is adaptive to different sensor characteristics, and the model complexity can be dynamically adjusted on-the-fly according to different data scales.