# Surface Normal Estimation with Transformers

arXiv: https://arxiv.org/abs/2401.05745 · HTML: https://mlsanity.com/paper/2401.05745/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 48 references that are in the map.
- First posted on arXiv 11 Jan 2024.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [An End-to-End Transformer Model for 3D Object Detection](https://mlsanity.com/paper/2109.08141/) (2021)
- [PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformers](https://mlsanity.com/paper/2108.08839/) (2021)
- [Semantic3D.net: A new Large-scale Point Cloud Classification Benchmark](https://mlsanity.com/paper/1704.03847/) (2017)
- [PCPNET: Learning Local Shape Properties from Raw Point Clouds](https://mlsanity.com/paper/1710.04954/) (2017)
- [UniT3D: A Unified Transformer for 3D Dense Captioning and Visual Grounding](https://mlsanity.com/paper/2212.00836/) (2022)
- [Nesti-Net: Normal Estimation for Unstructured 3D Point Clouds using Convolutional Neural Networks](https://mlsanity.com/paper/1812.00709/) (2018)
- [DeepFit: 3D Surface Fitting via Neural Network Weighted Least Squares](https://mlsanity.com/paper/2003.10826/) (2020)
- [NeAF: Learning Neural Angle Fields for Point Normal Estimation](https://mlsanity.com/paper/2211.16869/) (2022)
- [Deep Feature-preserving Normal Estimation for Point Cloud Filtering](https://mlsanity.com/paper/2004.11563/) (2020)
- [AdaFit: Rethinking Learning-based Normal Estimation on Point Clouds](https://mlsanity.com/paper/2108.05836/) (2021)
- [Refine-Net: Normal Refinement Neural Network for Noisy Point Clouds](https://mlsanity.com/paper/2203.12514/) (2022)
- [HSurf-Net: Normal Estimation for 3D Point Clouds by Learning Hyper Surfaces](https://mlsanity.com/paper/2210.07158/) (2022)
- [Normal Estimation for 3D Point Clouds via Local Plane Constraint and Multi-scale Selection](https://mlsanity.com/paper/1910.08537/) (2019)
- [GraphFit: Learning Multi-scale Graph-Convolutional Representation for Point Cloud Normal Estimation](https://mlsanity.com/paper/2207.11484/) (2022)
- [Improvement of Normal Estimation for PointClouds via Simplifying Surface Fitting](https://mlsanity.com/paper/2104.10369/) (2021)

## Built on by (most cited first, 1 of 1)

- [GeoStereo: A Unified Stereo Geometry Estimation Framework for Disparity and Surface Normal](https://mlsanity.com/paper/2607.24024/) (2026)

## Abstract (arXiv)

We propose the use of a Transformer to accurately predict normals from point clouds with noise and density variations. Previous learning-based methods utilize PointNet variants to explicitly extract multi-scale features at different input scales, then focus on a surface fitting method by which local point cloud neighborhoods are fitted to a geometric surface approximated by either a polynomial function or a multi-layer perceptron (MLP). However, fitting surfaces to fixed-order polynomial functions can suffer from overfitting or underfitting, and learning MLP-represented hyper-surfaces requires pre-generated per-point weights. To avoid these limitations, we first unify the design choices in previous works and then propose a simplified Transformer-based model to extract richer and more robust geometric features for the surface normal estimation task. Through extensive experiments, we demonstrate that our Transformer-based method achieves state-of-the-art performance on both the synthetic shape dataset PCPNet, and the real-world indoor scene dataset SceneNN, exhibiting more noise-resilient behavior and significantly faster inference. Most importantly, we demonstrate that the sophisticated hand-designed modules in existing works are not necessary to excel at the task of surface normal estimation.