# Low Latency Point Cloud Rendering with Learned Splatting

arXiv: https://arxiv.org/abs/2409.16504 · HTML: https://mlsanity.com/paper/2409.16504/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 36 references that are in the map.
- First posted on arXiv 24 Sep 2024.

## Builds on (most cited first, 11 of 11)

- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [Open3D: A Modern Library for 3D Data Processing](https://mlsanity.com/paper/1801.09847/) (2018)
- [BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networks](https://mlsanity.com/paper/1911.10127/) (2019)
- [Neural Point-Based Graphics](https://mlsanity.com/paper/1906.08240/) (2019)
- [Function4D: Real-time Human Volumetric Capture from Very Sparse Consumer RGBD Sensors](https://mlsanity.com/paper/2105.01859/) (2021)
- [Differentiable Surface Splatting for Point-based Geometry Processing](https://mlsanity.com/paper/1906.04173/) (2019)
- [Pointersect: Neural Rendering with Cloud-Ray Intersection](https://mlsanity.com/paper/2304.12390/) (2023)
- [Deep Multi Depth Panoramas for View Synthesis](https://mlsanity.com/paper/2008.01815/) (2020)

## Built on by (most cited first, 5 of 5)

- [GradiSeg: Gradient-Guided Gaussian Segmentation with Enhanced 3D Boundary Precision](https://mlsanity.com/paper/2412.00392/) (2024)
- [Learning-based 3D Reconstruction in Autonomous Driving: A Comprehensive Survey](https://mlsanity.com/paper/2503.14537/) (2025)
- [U-Motion: Learned Point Cloud Video Compression with U-Structured Temporal Context Generation](https://mlsanity.com/paper/2411.14501/) (2024)
- [TeSO: Representing and Compressing 3D Point Cloud Scenes with Textured Surfel Octree](https://mlsanity.com/paper/2508.07083/) (2025)
- [Optimization-Free Style Transfer for 3D Gaussian Splats](https://mlsanity.com/paper/2508.05813/) (2025)

## Abstract (arXiv)

Point cloud is a critical 3D representation with many emerging applications. Because of the point sparsity and irregularity, high-quality rendering of point clouds is challenging and often requires complex computations to recover the continuous surface representation. On the other hand, to avoid visual discomfort, the motion-to-photon latency has to be very short, under 10 ms. Existing rendering solutions lack in either quality or speed. To tackle these challenges, we present a framework that unlocks interactive, free-viewing and high-fidelity point cloud rendering. We train a generic neural network to estimate 3D elliptical Gaussians from arbitrary point clouds and use differentiable surface splatting to render smooth texture and surface normal for arbitrary views. Our approach does not require per-scene optimization, and enable real-time rendering of dynamic point cloud. Experimental results demonstrate the proposed solution enjoys superior visual quality and speed, as well as generalizability to different scene content and robustness to compression artifacts. The code is available at https://github.com/huzi96/gaussian-pcloud-render .