# ALTO: Alternating Latent Topologies for Implicit 3D Reconstruction

arXiv: https://arxiv.org/abs/2212.04096 · HTML: https://mlsanity.com/paper/2212.04096/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 70 of its 91 references that are in the map.
- First posted on arXiv 8 Dec 2022.

## Builds on (most cited first, 30 of 70)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [KPConv: Flexible and Deformable Convolution for Point Clouds](https://mlsanity.com/paper/1904.08889/) (2019)
- [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018)
- [A Point Set Generation Network for 3D Object Reconstruction from a Single Image](https://mlsanity.com/paper/1612.00603/) (2016)
- [Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations](https://mlsanity.com/paper/1906.01618/) (2019)
- [PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization](https://mlsanity.com/paper/1905.05172/) (2019)
- [MVSNet: Depth Inference for Unstructured Multi-view Stereo](https://mlsanity.com/paper/1804.02505/) (2018)
- [Convolutional Occupancy Networks](https://mlsanity.com/paper/2003.04618/) (2020)
- [Implicit Geometric Regularization for Learning Shapes](https://mlsanity.com/paper/2002.10099/) (2020)
- [Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision](https://mlsanity.com/paper/1912.07372/) (2019)
- [Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images](https://mlsanity.com/paper/1804.01654/) (2018)
- [3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction](https://mlsanity.com/paper/1604.00449/) (2016)
- [Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling](https://mlsanity.com/paper/1610.07584/) (2016)
- [AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generation](https://mlsanity.com/paper/1802.05384/) (2018)
- [PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization](https://mlsanity.com/paper/2004.00452/) (2020)
- [PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows](https://mlsanity.com/paper/1906.12320/) (2019)
- [Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes](https://mlsanity.com/paper/2101.10994/) (2021)
- [SAL: Sign Agnostic Learning of Shapes from Raw Data](https://mlsanity.com/paper/1911.10414/) (2019)

## Built on by (most cited first, 21 of 21)

- [Feature 3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled Feature Fields](https://mlsanity.com/paper/2312.03203/) (2023)
- [SpatialTracker: Tracking Any 2D Pixels in 3D Space](https://mlsanity.com/paper/2404.04319/) (2024)
- [4K4DGen: Panoramic 4D Generation at 4K Resolution](https://mlsanity.com/paper/2406.13527/) (2024)
- [SemCity: Semantic Scene Generation with Triplane Diffusion](https://mlsanity.com/paper/2403.07773/) (2024)
- [Fast Learning of Signed Distance Functions from Noisy Point Clouds via Noise to Noise Mapping](https://mlsanity.com/paper/2407.14225/) (2024)
- [EMIE-MAP: Large-Scale Road Surface Reconstruction Based on Explicit Mesh and Implicit Encoding](https://mlsanity.com/paper/2403.11789/) (2024)
- [Pushing the Limits of 3D Shape Generation at Scale](https://mlsanity.com/paper/2306.11510/) (2023)
- [SliceOcc: Indoor 3D Semantic Occupancy Prediction with Vertical Slice Representation](https://mlsanity.com/paper/2501.16684/) (2025)
- [Ins-HOI: Instance Aware Human-Object Interactions Recovery](https://mlsanity.com/paper/2312.09641/) (2023)
- [GridFormer: Point-Grid Transformer for Surface Reconstruction](https://mlsanity.com/paper/2401.02292/) (2024)
- [Implicit Filtering for Learning Neural Signed Distance Functions from 3D Point Clouds](https://mlsanity.com/paper/2407.13342/) (2024)
- [Inferring Neural Signed Distance Functions by Overfitting on Single Noisy Point Clouds through Finetuning Data-Driven based Priors](https://mlsanity.com/paper/2410.19680/) (2024)
- [Monge-Ampere Regularization for Learning Arbitrary Shapes from Point Clouds](https://mlsanity.com/paper/2410.18477/) (2024)
- [Learning Any-View 6DoF Robotic Grasping in Cluttered Scenes via Neural Surface Rendering](https://mlsanity.com/paper/2306.07392/) (2023)
- [PointDreamer: Zero-shot 3D Textured Mesh Reconstruction from Colored Point Cloud](https://mlsanity.com/paper/2406.15811/) (2024)
- [NoKSR: Kernel-Free Neural Surface Reconstruction via Point Cloud Serialization](https://mlsanity.com/paper/2502.12534/) (2025)
- [Sharpening Neural Implicit Functions with Frequency Consolidation Priors](https://mlsanity.com/paper/2412.19720/) (2024)
- [Reconstructing Building Height from Spaceborne TomoSAR Point Clouds Using a Dual-Topology Network](https://mlsanity.com/paper/2601.00658/) (2026)
- [DITTO: Dual and Integrated Latent Topologies for Implicit 3D Reconstruction](https://mlsanity.com/paper/2403.05005/) (2024)
- [Polyhedral Surface: Self-supervised Point Cloud Reconstruction Based on Polyhedral Surface](https://mlsanity.com/paper/2310.14560/) (2023)
- [SurfR: Surface Reconstruction with Multi-scale Attention](https://mlsanity.com/paper/2506.08635/) (2025)

## Abstract (arXiv)

This work introduces alternating latent topologies (ALTO) for high-fidelity reconstruction of implicit 3D surfaces from noisy point clouds. Previous work identifies that the spatial arrangement of latent encodings is important to recover detail. One school of thought is to encode a latent vector for each point (point latents). Another school of thought is to project point latents into a grid (grid latents) which could be a voxel grid or triplane grid. Each school of thought has tradeoffs. Grid latents are coarse and lose high-frequency detail. In contrast, point latents preserve detail. However, point latents are more difficult to decode into a surface, and quality and runtime suffer. In this paper, we propose ALTO to sequentially alternate between geometric representations, before converging to an easy-to-decode latent. We find that this preserves spatial expressiveness and makes decoding lightweight. We validate ALTO on implicit 3D recovery and observe not only a performance improvement over the state-of-the-art, but a runtime improvement of 3-10×. Project website at https://visual.ee.ucla.edu/alto.htm/.