# Rendering-Oriented 3D Point Cloud Attribute Compression using Sparse Tensor-based Transformer

arXiv: https://arxiv.org/abs/2411.07899 · HTML: https://mlsanity.com/paper/2411.07899/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 73 references that are in the map.
- First posted on arXiv 12 Nov 2024.

## Builds on (most cited first, 30 of 39)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Plenoxels: Radiance Fields without Neural Networks](https://mlsanity.com/paper/2112.05131/) (2021)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020)
- [PCT: Point cloud transformer](https://mlsanity.com/paper/2012.09688/) (2020)
- [End-to-end Optimized Image Compression](https://mlsanity.com/paper/1611.01704/) (2016)
- [Joint Autoregressive and Hierarchical Priors for Learned Image Compression](https://mlsanity.com/paper/1809.02736/) (2018)
- [Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modules](https://mlsanity.com/paper/2001.01568/) (2020)
- [Point Transformer V2: Grouped Vector Attention and Partition-based Pooling](https://mlsanity.com/paper/2210.05666/) (2022)
- [SynSin: End-to-end View Synthesis from a Single Image](https://mlsanity.com/paper/1912.08804/) (2019)
- [Differentiable Surface Splatting for Point-based Geometry Processing](https://mlsanity.com/paper/1906.04173/) (2019)
- [Deep Contextual Video Compression](https://mlsanity.com/paper/2109.15047/) (2021)
- [The Devil Is in the Details: Window-based Attention for Image Compression](https://mlsanity.com/paper/2203.08450/) (2022)
- [Learning Efficient Point Cloud Generation for Dense 3D Object Reconstruction](https://mlsanity.com/paper/1706.07036/) (2017)
- [Temporal Context Mining for Learned Video Compression](https://mlsanity.com/paper/2111.13850/) (2021)
- [Unsupervised Learning of Shape and Pose with Differentiable Point Clouds](https://mlsanity.com/paper/1810.09381/) (2018)
- [Multiscale Point Cloud Geometry Compression](https://mlsanity.com/paper/2011.03799/) (2020)
- [OctSqueeze: Octree-Structured Entropy Model for LiDAR Compression](https://mlsanity.com/paper/2005.07178/) (2020)
- [TorchSparse: Efficient Point Cloud Inference Engine](https://mlsanity.com/paper/2204.10319/) (2022)
- [Sparse Tensor-based Multiscale Representation for Point Cloud Geometry Compression](https://mlsanity.com/paper/2111.10633/) (2021)
- [Generative Sparse Detection Networks for 3D Single-shot Object Detection](https://mlsanity.com/paper/2006.12356/) (2020)
- [VoxelContext-Net: An Octree based Framework for Point Cloud Compression](https://mlsanity.com/paper/2105.02158/) (2021)
- [OctAttention: Octree-Based Large-Scale Contexts Model for Point Cloud Compression](https://mlsanity.com/paper/2202.06028/) (2022)
- [Sparse Tensor-based Point Cloud Attribute Compression](https://mlsanity.com/paper/2204.01023/) (2022)
- [Lossless Point Cloud Geometry and Attribute Compression Using a Learned Conditional Probability Model](https://mlsanity.com/paper/2303.06519/) (2023)
- [Point Cloud Rendering after Coding: Impacts on Subjective and Objective Quality](https://mlsanity.com/paper/1912.09137/) (2019)
- [Pointersect: Neural Rendering with Cloud-Ray Intersection](https://mlsanity.com/paper/2304.12390/) (2023)

## Built on by (most cited first, 4 of 4)

- [Deep Unrolling of Sparsity-Induced RDO for 3D Point Cloud Attribute Coding](https://mlsanity.com/paper/2509.08685/) (2025)
- [Squeeze3D: Extreme Neural Compression with Latent Space Bridging](https://mlsanity.com/paper/2506.07932/) (2025)
- [Rate-distortion Optimized Point Cloud Preprocessing for Geometry-based Point Cloud Compression](https://mlsanity.com/paper/2508.01633/) (2025)
- [DUGAE: Unified Geometry and Attribute Enhancement via Spatiotemporal Correlations for G-PCC Compressed Dynamic Point Clouds](https://mlsanity.com/paper/2603.26183/) (2026)

## Abstract (arXiv)

The evolution of 3D visualization techniques has fundamentally transformed how we interact with digital content. At the forefront of this change is point cloud technology, offering an immersive experience that surpasses traditional 2D representations. However, the massive data size of point clouds presents significant challenges in data compression. Current methods for lossy point cloud attribute compression (PCAC) generally focus on reconstructing the original point clouds with minimal error. However, for point cloud visualization scenarios, the reconstructed point clouds with distortion still need to undergo a complex rendering process, which affects the final user-perceived quality. In this paper, we propose an end-to-end deep learning framework that seamlessly integrates PCAC with differentiable rendering, denoted as rendering-oriented PCAC (RO-PCAC), directly targeting the quality of rendered multiview images for viewing. In a differentiable manner, the impact of the rendering process on the reconstructed point clouds is taken into account. Moreover, we characterize point clouds as sparse tensors and propose a sparse tensor-based transformer, called SP-Trans. By aligning with the local density of the point cloud and utilizing an enhanced local attention mechanism, SP-Trans captures the intricate relationships within the point cloud, further improving feature analysis and synthesis within the framework. Extensive experiments demonstrate that the proposed RO-PCAC achieves state-of-the-art compression performance, compared to existing reconstruction-oriented methods, including traditional, learning-based, and hybrid methods.