# LEAP: Liberate Sparse-view 3D Modeling from Camera Poses

arXiv: https://arxiv.org/abs/2310.01410 · HTML: https://mlsanity.com/paper/2310.01410/

## Key facts

- Cited by 65 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 43 references that are in the map.
- First posted on arXiv 2 Oct 2023.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [Plenoxels: Radiance Fields without Neural Networks](https://mlsanity.com/paper/2112.05131/) (2021)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [Objaverse: A Universe of Annotated 3D Objects](https://mlsanity.com/paper/2212.08051/) (2022)
- [Zero-1-to-3: Zero-shot One Image to 3D Object](https://mlsanity.com/paper/2303.11328/) (2023)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo](https://mlsanity.com/paper/2103.15595/) (2021)
- [IBRNet: Learning Multi-View Image-Based Rendering](https://mlsanity.com/paper/2102.13090/) (2021)
- [BARF: Bundle-Adjusting Neural Radiance Fields](https://mlsanity.com/paper/2104.06405/) (2021)
- [RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputs](https://mlsanity.com/paper/2112.00724/) (2021)
- [Kubric: A scalable dataset generator](https://mlsanity.com/paper/2203.03570/) (2022)
- [OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generation](https://mlsanity.com/paper/2301.07525/) (2023)
- [FreeNeRF: Improving Few-shot Neural Rendering with Free Frequency Regularization](https://mlsanity.com/paper/2303.07418/) (2023)
- [Vector Neurons: A General Framework for SO(3)-Equivariant Networks](https://mlsanity.com/paper/2104.12229/) (2021)
- [A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondence](https://mlsanity.com/paper/2305.15347/) (2023)
- [Scene Representation Transformer: Geometry-Free Novel View Synthesis Through Set-Latent Scene Representations](https://mlsanity.com/paper/2111.13152/) (2021)
- [SPARF: Neural Radiance Fields from Sparse and Noisy Poses](https://mlsanity.com/paper/2211.11738/) (2022)
- [NeRS: Neural Reflectance Surfaces for Sparse-view 3D Reconstruction in the Wild](https://mlsanity.com/paper/2110.07604/) (2021)
- [RelPose: Predicting Probabilistic Relative Rotation for Single Objects in the Wild](https://mlsanity.com/paper/2208.05963/) (2022)
- [RelPose++: Recovering 6D Poses from Sparse-view Observations](https://mlsanity.com/paper/2305.04926/) (2023)
- [SparsePose: Sparse-View Camera Pose Regression and Refinement](https://mlsanity.com/paper/2211.16991/) (2022)
- [Few-View Object Reconstruction with Unknown Categories and Camera Poses](https://mlsanity.com/paper/2212.04492/) (2022)

## Built on by (most cited first, 30 of 65)

- [GS-LRM: Large Reconstruction Model for 3D Gaussian Splatting](https://mlsanity.com/paper/2404.19702/) (2024)
- [LVSM: A Large View Synthesis Model with Minimal 3D Inductive Bias](https://mlsanity.com/paper/2410.17242/) (2024)
- [FLARE: Feed-forward Geometry, Appearance and Camera Estimation from Uncalibrated Sparse Views](https://mlsanity.com/paper/2502.12138/) (2025)
- [Splatt3R: Zero-shot Gaussian Splatting from Uncalibrated Image Pairs](https://mlsanity.com/paper/2408.13912/) (2024)
- [SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View Consistency](https://mlsanity.com/paper/2407.17470/) (2024)
- [Era3D: High-Resolution Multiview Diffusion using Efficient Row-wise Attention](https://mlsanity.com/paper/2405.11616/) (2024)
- [MVDiffusion++: A Dense High-resolution Multi-view Diffusion Model for Single or Sparse-view 3D Object Reconstruction](https://mlsanity.com/paper/2402.12712/) (2024)
- [RayZer: A Self-supervised Large View Synthesis Model](https://mlsanity.com/paper/2505.00702/) (2025)
- [WorldMirror: Universal 3D World Reconstruction with Any-Prior Prompting](https://mlsanity.com/paper/2510.10726/) (2025)
- [PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting](https://mlsanity.com/paper/2410.22128/) (2024)
- [GaussianObject: High-Quality 3D Object Reconstruction from Four Views with Gaussian Splatting](https://mlsanity.com/paper/2402.10259/) (2024)
- [OmniGlue: Generalizable Feature Matching with Foundation Model Guidance](https://mlsanity.com/paper/2405.12979/) (2024)
- [SV4D 2.0: Enhancing Spatio-Temporal Consistency in Multi-View Video Diffusion for High-Quality 4D Generation](https://mlsanity.com/paper/2503.16396/) (2025)
- [VideoMV: Consistent Multi-View Generation Based on Large Video Generative Model](https://mlsanity.com/paper/2403.12010/) (2024)
- [FreeSplatter: Pose-free Gaussian Splatting for Sparse-view 3D Reconstruction](https://mlsanity.com/paper/2412.09573/) (2024)
- [No Pose at All: Self-Supervised Pose-Free 3D Gaussian Splatting from Sparse Views](https://mlsanity.com/paper/2508.01171/) (2025)
- [Cycle3D: High-quality and Consistent Image-to-3D Generation via Generation-Reconstruction Cycle](https://mlsanity.com/paper/2407.19548/) (2024)
- [SpaRP: Fast 3D Object Reconstruction and Pose Estimation from Sparse Views](https://mlsanity.com/paper/2408.10195/) (2024)
- [ReconViaGen: Towards Accurate Multi-view 3D Object Reconstruction via Generation](https://mlsanity.com/paper/2510.23306/) (2025)
- [Ouroboros3D: Image-to-3D Generation via 3D-aware Recursive Diffusion](https://mlsanity.com/paper/2406.03184/) (2024)
- [Advances in Feed-Forward 3D Reconstruction and View Synthesis: A Survey](https://mlsanity.com/paper/2507.14501/) (2025)
- [Real3D: Scaling Up Large Reconstruction Models with Real-World Images](https://mlsanity.com/paper/2406.08479/) (2024)
- [UFORecon: Generalizable Sparse-View Surface Reconstruction from Arbitrary and UnFavOrable Sets](https://mlsanity.com/paper/2403.05086/) (2024)
- [E-RayZer: Self-supervised 3D Reconstruction as Spatial Visual Pre-training](https://mlsanity.com/paper/2512.10950/) (2025)
- [MegaSynth: Scaling Up 3D Scene Reconstruction with Synthesized Data](https://mlsanity.com/paper/2412.14166/) (2024)
- [Epipolar-Free 3D Gaussian Splatting for Generalizable Novel View Synthesis](https://mlsanity.com/paper/2410.22817/) (2024)
- [Atlas Gaussians Diffusion for 3D Generation](https://mlsanity.com/paper/2408.13055/) (2024)
- [SymmCompletion: High-Fidelity and High-Consistency Point Cloud Completion with Symmetry Guidance](https://mlsanity.com/paper/2503.18007/) (2025)
- [iFusion: Inverting Diffusion for Pose-Free Reconstruction from Sparse Views](https://mlsanity.com/paper/2312.17250/) (2023)
- [TokenSplat: Token-aligned 3D Gaussian Splatting for Feed-forward Pose-free Reconstruction](https://mlsanity.com/paper/2603.00697/) (2026)

## Abstract (arXiv)

Are camera poses necessary for multi-view 3D modeling? Existing approaches predominantly assume access to accurate camera poses. While this assumption might hold for dense views, accurately estimating camera poses for sparse views is often elusive. Our analysis reveals that noisy estimated poses lead to degraded performance for existing sparse-view 3D modeling methods. To address this issue, we present LEAP, a novel pose-free approach, therefore challenging the prevailing notion that camera poses are indispensable. LEAP discards pose-based operations and learns geometric knowledge from data. LEAP is equipped with a neural volume, which is shared across scenes and is parameterized to encode geometry and texture priors. For each incoming scene, we update the neural volume by aggregating 2D image features in a feature-similarity-driven manner. The updated neural volume is decoded into the radiance field, enabling novel view synthesis from any viewpoint. On both object-centric and scene-level datasets, we show that LEAP significantly outperforms prior methods when they employ predicted poses from state-of-the-art pose estimators. Notably, LEAP performs on par with prior approaches that use ground-truth poses while running 400× faster than PixelNeRF. We show LEAP generalizes to novel object categories and scenes, and learns knowledge closely resembles epipolar geometry. Project page: https://hwjiang1510.github.io/LEAP/