# PointGMM: a Neural GMM Network for Point Clouds

arXiv: https://arxiv.org/abs/2003.13326 · HTML: https://mlsanity.com/paper/2003.13326/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Mar 2020.

## Built on by (most cited first, 30 of 33)

- [UnsupervisedR&R: Unsupervised Point Cloud Registration via Differentiable Rendering](https://mlsanity.com/paper/2102.11870/) (2021)
- [Approximate Differentiable Rendering with Algebraic Surfaces](https://mlsanity.com/paper/2207.10606/) (2022)
- [Relaxing Accurate Initialization Constraint for 3D Gaussian Splatting](https://mlsanity.com/paper/2403.09413/) (2024)
- [Orienting Point Clouds with Dipole Propagation](https://mlsanity.com/paper/2105.01604/) (2021)
- [Continual Neural Mapping: Learning An Implicit Scene Representation from Sequential Observations](https://mlsanity.com/paper/2108.05851/) (2021)
- [SPAGHETTI: Editing Implicit Shapes Through Part Aware Generation](https://mlsanity.com/paper/2201.13168/) (2022)
- [Flexible Techniques for Differentiable Rendering with 3D Gaussians](https://mlsanity.com/paper/2308.14737/) (2023)
- [Bootstrap Your Own Correspondences](https://mlsanity.com/paper/2106.00677/) (2021)
- [Doodle Your 3D: From Abstract Freehand Sketches to Precise 3D Shapes](https://mlsanity.com/paper/2312.04043/) (2023)
- [CNS-Edit: 3D Shape Editing via Coupled Neural Shape Optimization](https://mlsanity.com/paper/2402.02313/) (2024)
- [Full Transformer Framework for Robust Point Cloud Registration with Deep Information Interaction](https://mlsanity.com/paper/2112.09385/) (2021)
- [Point Cloud Registration for LiDAR and Photogrammetric Data: a Critical Synthesis and Performance Analysis on Classic and Deep Learning Algorithms](https://mlsanity.com/paper/2302.07184/) (2023)
- [Stein ICP for Uncertainty Estimation in Point Cloud Matching](https://mlsanity.com/paper/2106.03287/) (2021)
- [Pix4Point: Image Pretrained Standard Transformers for 3D Point Cloud Understanding](https://mlsanity.com/paper/2208.12259/) (2022)
- [Self-Supervised Correspondence Estimation via Multiview Registration](https://mlsanity.com/paper/2212.03236/) (2022)
- [MM-Point: Multi-View Information-Enhanced Multi-Modal Self-Supervised 3D Point Cloud Understanding](https://mlsanity.com/paper/2402.10002/) (2024)
- [RPG: Learning Recursive Point Cloud Generation](https://mlsanity.com/paper/2105.14322/) (2021)
- [MGAug: Multimodal Geometric Augmentation in Latent Spaces of Image Deformations](https://mlsanity.com/paper/2312.13440/) (2023)
- [WIR3D: Visually-Informed and Geometry-Aware 3D Shape Abstraction](https://mlsanity.com/paper/2505.04813/) (2025)
- [Z2P: Instant Visualization of Point Clouds](https://mlsanity.com/paper/2105.14548/) (2021)
- [SAT: Size-Aware Transformer for 3D Point Cloud Semantic Segmentation](https://mlsanity.com/paper/2301.06869/) (2023)
- [Advancements in Point Cloud-Based 3D Defect Detection and Classification for Industrial Systems: A Comprehensive Survey](https://mlsanity.com/paper/2402.12923/) (2024)
- [(Just) A Spoonful of Refinements Helps the Registration Error Go Down](https://mlsanity.com/paper/2108.03257/) (2021)
- [DeepBBS: Deep Best Buddies for Point Cloud Registration](https://mlsanity.com/paper/2110.03016/) (2021)
- [Online Learning of a Probabilistic and Adaptive Scene Representation](https://mlsanity.com/paper/2103.16832/) (2021)
- [Parameterize Structure with Differentiable Template for 3D Shape Generation](https://mlsanity.com/paper/2410.10399/) (2024)
- [Localized Shape Modelling with Global Coherence: An Inverse Spectral Approach](https://mlsanity.com/paper/2108.02161/) (2021)
- [Rotation Equivariant 3D Hand Mesh Generation from a Single RGB Image](https://mlsanity.com/paper/2111.13023/) (2021)
- [Stress-Testing Point Cloud Registration on Automotive LiDAR](https://mlsanity.com/paper/2204.07719/) (2022)
- [V2I-Calib: A Novel Calibration Approach for Collaborative Vehicle and Infrastructure LiDAR Systems](https://mlsanity.com/paper/2407.10195/) (2024)

## Abstract (arXiv)

Point clouds are a popular representation for 3D shapes. However, they encode a particular sampling without accounting for shape priors or non-local information. We advocate for the use of a hierarchical Gaussian mixture model (hGMM), which is a compact, adaptive and lightweight representation that probabilistically defines the underlying 3D surface. We present PointGMM, a neural network that learns to generate hGMMs which are characteristic of the shape class, and also coincide with the input point cloud. PointGMM is trained over a collection of shapes to learn a class-specific prior. The hierarchical representation has two main advantages: (i) coarse-to-fine learning, which avoids converging to poor local-minima; and (ii) (an unsupervised) consistent partitioning of the input shape. We show that as a generative model, PointGMM learns a meaningful latent space which enables generating consistent interpolations between existing shapes, as well as synthesizing novel shapes. We also present a novel framework for rigid registration using PointGMM, that learns to disentangle orientation from structure of an input shape.