# PU-Ray: Domain-Independent Point Cloud Upsampling via Ray Marching on Neural Implicit Surface

arXiv: https://arxiv.org/abs/2310.08755 · HTML: https://mlsanity.com/paper/2310.08755/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 58 references that are in the map.
- First posted on arXiv 12 Oct 2023.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [PCT: Point cloud transformer](https://mlsanity.com/paper/2012.09688/) (2020)
- [KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D](https://mlsanity.com/paper/2109.13410/) (2021)
- [Deep Learning for 3D Point Clouds: A Survey](https://mlsanity.com/paper/1912.12033/) (2019)
- [Neural Unsigned Distance Fields for Implicit Function Learning](https://mlsanity.com/paper/2010.13938/) (2020)
- [PU-Net: Point Cloud Upsampling Network](https://mlsanity.com/paper/1801.06761/) (2018)
- [DIST: Rendering Deep Implicit Signed Distance Function with Differentiable Sphere Tracing](https://mlsanity.com/paper/1911.13225/) (2019)
- [PU-GAN: a Point Cloud Upsampling Adversarial Network](https://mlsanity.com/paper/1907.10844/) (2019)
- [Deep Learning for Image and Point Cloud Fusion in Autonomous Driving: A Review](https://mlsanity.com/paper/2004.05224/) (2020)
- [Patch-based Progressive 3D Point Set Upsampling](https://mlsanity.com/paper/1811.11286/) (2018)
- [PU-GCN: Point Cloud Upsampling using Graph Convolutional Networks](https://mlsanity.com/paper/1912.03264/) (2019)
- [VISTA 2.0: An Open, Data-driven Simulator for Multimodal Sensing and Policy Learning for Autonomous Vehicles](https://mlsanity.com/paper/2111.12083/) (2021)
- [Point Cloud Upsampling via Disentangled Refinement](https://mlsanity.com/paper/2106.04779/) (2021)
- [PUGeo-Net: A Geometry-centric Network for 3D Point Cloud Upsampling](https://mlsanity.com/paper/2002.10277/) (2020)
- [Neural Points: Point Cloud Representation with Neural Fields for Arbitrary Upsampling](https://mlsanity.com/paper/2112.04148/) (2021)
- [Comprehensive Review of Deep Learning-Based 3D Point Cloud Completion Processing and Analysis](https://mlsanity.com/paper/2203.03311/) (2022)
- [Grad-PU: Arbitrary-Scale Point Cloud Upsampling via Gradient Descent with Learned Distance Functions](https://mlsanity.com/paper/2304.11846/) (2023)
- [Deep Implicit Surface Point Prediction Networks](https://mlsanity.com/paper/2106.05779/) (2021)
- [Meta-PU: An Arbitrary-Scale Upsampling Network for Point Cloud](https://mlsanity.com/paper/2102.04317/) (2021)
- [PU-Transformer: Point Cloud Upsampling Transformer](https://mlsanity.com/paper/2111.12242/) (2021)
- [SPU-Net: Self-Supervised Point Cloud Upsampling by Coarse-to-Fine Reconstruction with Self-Projection Optimization](https://mlsanity.com/paper/2012.04439/) (2020)
- [Self-Supervised Arbitrary-Scale Point Clouds Upsampling via Implicit Neural Representation](https://mlsanity.com/paper/2204.08196/) (2022)
- [Simulation-based Lidar Super-resolution for Ground Vehicles](https://mlsanity.com/paper/2004.05242/) (2020)
- [SSPU-Net: Self-Supervised Point Cloud Upsampling via Differentiable Rendering](https://mlsanity.com/paper/2108.00454/) (2021)
- [Implicit LiDAR Network: LiDAR Super-Resolution via Interpolation Weight Prediction](https://mlsanity.com/paper/2203.06413/) (2022)
- [Pseudo-LiDAR Point Cloud Interpolation Based on 3D Motion Representation and Spatial Supervision](https://mlsanity.com/paper/2006.11481/) (2020)
- [Lidar Upsampling with Sliced Wasserstein Distance](https://mlsanity.com/paper/2301.13558/) (2023)

## Built on by (most cited first, 1 of 1)

- [2D Representation for Unguided Single-View 3D Super-Resolution in Real-Time](https://mlsanity.com/paper/2511.08224/) (2025)

## Abstract (arXiv)

While recent advancements in deep-learning point cloud upsampling methods have improved the input to intelligent transportation systems, they still suffer from issues of domain dependency between synthetic and real-scanned point clouds. This paper addresses the above issues by proposing a new ray-based upsampling approach with an arbitrary rate, where a depth prediction is made for each query ray and its corresponding patch. Our novel method simulates the sphere-tracing ray marching algorithm on the neural implicit surface defined with an unsigned distance function (UDF) to achieve more precise and stable ray-depth predictions by training a point-transformer-based network. The rule-based mid-point query sampling method generates more evenly distributed points without requiring an end-to-end model trained using a nearest-neighbor-based reconstruction loss function, which may be biased towards the training dataset. Self-supervised learning becomes possible with accurate ground truths within the input point cloud. The results demonstrate the method's versatility across domains and training scenarios with limited computational resources and training data. Comprehensive analyses of synthetic and real-scanned applications provide empirical evidence for the significance of the upsampling task across the computer vision and graphics domains to real-world applications of ITS.