# PU-Transformer: Point Cloud Upsampling Transformer

arXiv: https://arxiv.org/abs/2111.12242 · HTML: https://mlsanity.com/paper/2111.12242/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 54 of its 73 references that are in the map.
- First posted on arXiv 24 Nov 2021.

## Builds on (most cited first, 30 of 54)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Transformers in Vision: A Survey](https://mlsanity.com/paper/2101.01169/) (2021)
- [Accurate Image Super-Resolution Using Very Deep Convolutional Networks](https://mlsanity.com/paper/1511.04587/) (2015)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [PCT: Point cloud transformer](https://mlsanity.com/paper/2012.09688/) (2020)
- [Deep Hough Voting for 3D Object Detection in Point Clouds](https://mlsanity.com/paper/1904.09664/) (2019)
- [RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds](https://mlsanity.com/paper/1911.11236/) (2019)
- [ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks](https://mlsanity.com/paper/1910.03151/) (2019)
- [Multi-view Convolutional Neural Networks for 3D Shape Recognition](https://mlsanity.com/paper/1505.00880/) (2015)
- [Deep Learning for 3D Point Clouds: A Survey](https://mlsanity.com/paper/1912.12033/) (2019)
- [Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Data](https://mlsanity.com/paper/1908.04616/) (2019)
- [DeepGCNs: Can GCNs Go as Deep as CNNs?](https://mlsanity.com/paper/1904.03751/) (2019)
- [Joint 2D-3D-Semantic Data for Indoor Scene Understanding](https://mlsanity.com/paper/1702.01105/) (2017)

## Built on by (most cited first, 30 of 32)

- [Grad-PU: Arbitrary-Scale Point Cloud Upsampling via Gradient Descent with Learned Distance Functions](https://mlsanity.com/paper/2304.11846/) (2023)
- [Transformers in 3D Point Clouds: A Survey](https://mlsanity.com/paper/2205.07417/) (2022)
- [3DCTN: 3D Convolution-Transformer Network for Point Cloud Classification](https://mlsanity.com/paper/2203.00828/) (2022)
- [LCPFormer: Towards Effective 3D Point Cloud Analysis via Local Context Propagation in Transformers](https://mlsanity.com/paper/2210.12755/) (2022)
- [P2C: Self-Supervised Point Cloud Completion from Single Partial Clouds](https://mlsanity.com/paper/2307.14726/) (2023)
- [AGConv: Adaptive Graph Convolution on 3D Point Clouds](https://mlsanity.com/paper/2206.04665/) (2022)
- [TULIP: Transformer for Upsampling of LiDAR Point Clouds](https://mlsanity.com/paper/2312.06733/) (2023)
- [PointCaM: Cut-and-Mix for Open-Set Point Cloud Learning](https://mlsanity.com/paper/2212.02011/) (2022)
- [PointNSP: Autoregressive 3D Point Cloud Generation with Next-Scale Level-of-Detail Prediction](https://mlsanity.com/paper/2510.05613/) (2025)
- [Arbitrary Point Cloud Upsampling with Spherical Mixture of Gaussians](https://mlsanity.com/paper/2208.05274/) (2022)
- [Arbitrary-Scale Point Cloud Upsampling by Voxel-Based Network with Latent Geometric-Consistent Learning](https://mlsanity.com/paper/2403.05117/) (2024)
- [DiffPMAE: Diffusion Masked Autoencoders for Point Cloud Reconstruction](https://mlsanity.com/paper/2312.03298/) (2023)
- [Deep Learning for 3D Point Cloud Enhancement: A Survey](https://mlsanity.com/paper/2411.00857/) (2024)
- [Joint Point Cloud Upsampling and Cleaning with Octree-based CNNs](https://mlsanity.com/paper/2410.17001/) (2024)
- [PU-MFA : Point Cloud Up-sampling via Multi-scale Features Attention](https://mlsanity.com/paper/2208.10968/) (2022)
- [CU-Net: Real-Time High-Fidelity Color Upsampling for Point Clouds](https://mlsanity.com/paper/2209.06112/) (2022)
- [PillarGen: Enhancing Radar Point Cloud Density and Quality via Pillar-based Point Generation Network](https://mlsanity.com/paper/2403.01663/) (2024)
- [Rethinking Data Input for Point Cloud Upsampling](https://mlsanity.com/paper/2407.04476/) (2024)
- [NumGrad-Pull: Numerical Gradient Guided Tri-plane Representation for Surface Reconstruction from Point Clouds](https://mlsanity.com/paper/2411.17392/) (2024)
- [Parametric Surface Constrained Upsampler Network for Point Cloud](https://mlsanity.com/paper/2303.08240/) (2023)
- [PU-Ray: Domain-Independent Point Cloud Upsampling via Ray Marching on Neural Implicit Surface](https://mlsanity.com/paper/2310.08755/) (2023)
- [MBPU: A Plug-and-Play State Space Model for Point Cloud Upsamping with Fast Point Rendering](https://mlsanity.com/paper/2410.15941/) (2024)
- [Representation Learning of Point Cloud Upsampling in Global and Local Inputs](https://mlsanity.com/paper/2501.07076/) (2025)
- [SPU-IMR: Self-supervised Arbitrary-scale Point Cloud Upsampling via Iterative Mask-recovery Network](https://mlsanity.com/paper/2502.19452/) (2025)
- [PU-Gaussian: Point Cloud Upsampling using 3D Gaussian Representation](https://mlsanity.com/paper/2509.20207/) (2025)
- [Enhancing Sampling Protocol for Point Cloud Classification Against Corruptions](https://mlsanity.com/paper/2408.12062/) (2024)
- [PointNSP: Autoregressive 3D Point Cloud Generation with Next-Scale Level-of-Detail Prediction](https://mlsanity.com/paper/2503.08594/) (2025)
- [Preserving Topological and Geometric Embeddings for Point Cloud Recovery](https://mlsanity.com/paper/2507.19121/) (2025)
- [Point upsampling networks for single-photon sensing](https://mlsanity.com/paper/2508.12986/) (2025)
- [PUFM++: Point Cloud Upsampling via Enhanced Flow Matching](https://mlsanity.com/paper/2512.20988/) (2025)

## Abstract (arXiv)

Given the rapid development of 3D scanners, point clouds are becoming popular in AI-driven machines. However, point cloud data is inherently sparse and irregular, causing significant difficulties for machine perception. In this work, we focus on the point cloud upsampling task that intends to generate dense high-fidelity point clouds from sparse input data. Specifically, to activate the transformer's strong capability in representing features, we develop a new variant of a multi-head self-attention structure to enhance both point-wise and channel-wise relations of the feature map. In addition, we leverage a positional fusion block to comprehensively capture the local context of point cloud data, providing more position-related information about the scattered points. As the first transformer model introduced for point cloud upsampling, we demonstrate the outstanding performance of our approach by comparing with the state-of-the-art CNN-based methods on different benchmarks quantitatively and qualitatively.