# CU-Net: Real-Time High-Fidelity Color Upsampling for Point Clouds

arXiv: https://arxiv.org/abs/2209.06112 · HTML: https://mlsanity.com/paper/2209.06112/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 48 references that are in the map.
- First posted on arXiv 12 Sep 2022.

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [PCT: Point cloud transformer](https://mlsanity.com/paper/2012.09688/) (2020)
- [PU-Net: Point Cloud Upsampling Network](https://mlsanity.com/paper/1801.06761/) (2018)
- [FaceScape: a Large-scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Prediction](https://mlsanity.com/paper/2003.13989/) (2020)
- [PU-GAN: a Point Cloud Upsampling Adversarial Network](https://mlsanity.com/paper/1907.10844/) (2019)
- [Patch-based Progressive 3D Point Set Upsampling](https://mlsanity.com/paper/1811.11286/) (2018)
- [PU-GCN: Point Cloud Upsampling using Graph Convolutional Networks](https://mlsanity.com/paper/1912.03264/) (2019)
- [Point Cloud Upsampling via Disentangled Refinement](https://mlsanity.com/paper/2106.04779/) (2021)
- [TorchSparse: Efficient Point Cloud Inference Engine](https://mlsanity.com/paper/2204.10319/) (2022)
- [Neural Points: Point Cloud Representation with Neural Fields for Arbitrary Upsampling](https://mlsanity.com/paper/2112.04148/) (2021)
- [Meta-PU: An Arbitrary-Scale Upsampling Network for Point Cloud](https://mlsanity.com/paper/2102.04317/) (2021)
- [PU-Transformer: Point Cloud Upsampling Transformer](https://mlsanity.com/paper/2111.12242/) (2021)
- [Deep Magnification-Flexible Upsampling over 3D Point Clouds](https://mlsanity.com/paper/2011.12745/) (2020)
- [VirtualCube: An Immersive 3D Video Communication System](https://mlsanity.com/paper/2112.06730/) (2021)
- [Point Cloud Colorization Based on Densely Annotated 3D Shape Dataset](https://mlsanity.com/paper/1810.05396/) (2018)
- [Frequency-Selective Mesh-to-Mesh Resampling for Color Upsampling of Point Clouds](https://mlsanity.com/paper/2203.09224/) (2022)

## Built on by (most cited first, 2 of 2)

- [Hierarchical Prior-based Super Resolution for Point Cloud Geometry Compression](https://mlsanity.com/paper/2402.11250/) (2024)
- [Deep Learning based Joint Geometry and Attribute Up-sampling for Large-Scale Colored Point Clouds](https://mlsanity.com/paper/2506.22749/) (2025)

## Abstract (arXiv)

Point cloud upsampling is essential for high-quality augmented reality, virtual reality, and telepresence applications, due to the capture, processing, and communication limitations of existing technologies. Although geometry upsampling to densify a point cloud's coordinates has been well studied, the upsampling of the color attributes has been largely overlooked. In this paper, we propose CU-Net, the first deep-learning point cloud color upsampling model that enables low latency and high visual fidelity operation. CU-Net achieves linear time and space complexity by leveraging a feature extractor based on sparse convolution and a color prediction module based on neural implicit function. Therefore, CU-Net is theoretically guaranteed to be more efficient than most existing methods with quadratic complexity. Experimental results demonstrate that CU-Net can colorize a photo-realistic point cloud with nearly a million points in real time, while having notably better visual performance than baselines. Besides, CU-Net can adapt to arbitrary upsampling ratios and unseen objects without retraining. Our source code is available at https://github.com/UMass-LIDS/cunet.