# Progressively Optimized Local Radiance Fields for Robust View Synthesis

arXiv: https://arxiv.org/abs/2303.13791 · HTML: https://mlsanity.com/paper/2303.13791/

## Key facts

- Cited by 103 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 58 references that are in the map.
- First posted on arXiv 24 Mar 2023.

## Builds on (most cited first, 30 of 42)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [Plenoxels: Radiance Fields without Neural Networks](https://mlsanity.com/paper/2112.05131/) (2021)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [TensoRF: Tensorial Radiance Fields](https://mlsanity.com/paper/2203.09517/) (2022)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction](https://mlsanity.com/paper/2111.11215/) (2021)
- [NeRF++: Analyzing and Improving Neural Radiance Fields](https://mlsanity.com/paper/2010.07492/) (2020)
- [ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras](https://mlsanity.com/paper/1610.06475/) (2016)
- [Unsupervised Learning of Depth and Ego-Motion from Video](https://mlsanity.com/paper/1704.07813/) (2017)
- [Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes](https://mlsanity.com/paper/2011.13084/) (2020)
- [Block-NeRF: Scalable Large Scene Neural View Synthesis](https://mlsanity.com/paper/2202.05263/) (2022)
- [DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras](https://mlsanity.com/paper/2108.10869/) (2021)
- [BARF: Bundle-Adjusting Neural Radiance Fields](https://mlsanity.com/paper/2104.06405/) (2021)
- [RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputs](https://mlsanity.com/paper/2112.00724/) (2021)
- [NICE-SLAM: Neural Implicit Scalable Encoding for SLAM](https://mlsanity.com/paper/2112.12130/) (2021)
- [Dynamic View Synthesis from Dynamic Monocular Video](https://mlsanity.com/paper/2105.06468/) (2021)
- [Space-time Neural Irradiance Fields for Free-Viewpoint Video](https://mlsanity.com/paper/2011.12950/) (2020)
- [Mega-NeRF: Scalable Construction of Large-Scale NeRFs for Virtual Fly-Throughs](https://mlsanity.com/paper/2112.10703/) (2021)
- [GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose](https://mlsanity.com/paper/1803.02276/) (2018)
- [NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw Images](https://mlsanity.com/paper/2111.13679/) (2021)
- [Urban Radiance Fields](https://mlsanity.com/paper/2111.14643/) (2021)

## Built on by (most cited first, 30 of 103)

- [DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenes](https://mlsanity.com/paper/2312.07920/) (2023)
- [A Hierarchical 3D Gaussian Representation for Real-Time Rendering of Very Large Datasets](https://mlsanity.com/paper/2406.12080/) (2024)
- [COLMAP-Free 3D Gaussian Splatting](https://mlsanity.com/paper/2312.07504/) (2023)
- [Difix3D+: Improving 3D Reconstructions with Single-Step Diffusion Models](https://mlsanity.com/paper/2503.01774/) (2025)
- [StreetSurf: Extending Multi-view Implicit Surface Reconstruction to Street Views](https://mlsanity.com/paper/2306.04988/) (2023)
- [Consistent View Synthesis with Pose-Guided Diffusion Models](https://mlsanity.com/paper/2303.17598/) (2023)
- [VR-NeRF: High-Fidelity Virtualized Walkable Spaces](https://mlsanity.com/paper/2311.02542/) (2023)
- [InstantSplat: Sparse-view Gaussian Splatting in Seconds](https://mlsanity.com/paper/2403.20309/) (2024)
- [PLGSLAM: Progressive Neural Scene Represenation with Local to Global Bundle Adjustment](https://mlsanity.com/paper/2312.09866/) (2023)
- [SMERF: Streamable Memory Efficient Radiance Fields for Real-Time Large-Scene Exploration](https://mlsanity.com/paper/2312.07541/) (2023)
- [FlowMap: High-Quality Camera Poses, Intrinsics, and Depth via Gradient Descent](https://mlsanity.com/paper/2404.15259/) (2024)
- [CamP: Camera Preconditioning for Neural Radiance Fields](https://mlsanity.com/paper/2308.10902/) (2023)
- [On-the-fly Reconstruction for Large-Scale Novel View Synthesis from Unposed Images](https://mlsanity.com/paper/2506.05558/) (2025)
- [LU-NeRF: Scene and Pose Estimation by Synchronizing Local Unposed NeRFs](https://mlsanity.com/paper/2306.05410/) (2023)
- [FlowR: Flowing from Sparse to Dense 3D Reconstructions](https://mlsanity.com/paper/2504.01647/) (2025)
- [Feat2GS: Probing Visual Foundation Models with Gaussian Splatting](https://mlsanity.com/paper/2412.09606/) (2024)
- [A Construct-Optimize Approach to Sparse View Synthesis without Camera Pose](https://mlsanity.com/paper/2405.03659/) (2024)
- [LM-Gaussian: Boost Sparse-view 3D Gaussian Splatting with Large Model Priors](https://mlsanity.com/paper/2409.03456/) (2024)
- [LongSplat: Robust Unposed 3D Gaussian Splatting for Casual Long Videos](https://mlsanity.com/paper/2508.14041/) (2025)
- [Fast View Synthesis of Casual Videos with Soup-of-Planes](https://mlsanity.com/paper/2312.02135/) (2023)
- [Neural Fields in Robotics: A Survey](https://mlsanity.com/paper/2410.20220/) (2024)
- [AuraFusion360: Augmented Unseen Region Alignment for Reference-based 360deg Unbounded Scene Inpainting](https://mlsanity.com/paper/2502.05176/) (2025)
- [GGRt: Towards Pose-free Generalizable 3D Gaussian Splatting in Real-time](https://mlsanity.com/paper/2403.10147/) (2024)
- [NeRF-XL: Scaling NeRFs with Multiple GPUs](https://mlsanity.com/paper/2404.16221/) (2024)
- [Radar Fields: Frequency-Space Neural Scene Representations for FMCW Radar](https://mlsanity.com/paper/2405.04662/) (2024)
- [FrugalNeRF: Fast Convergence for Extreme Few-shot Novel View Synthesis without Learned Priors](https://mlsanity.com/paper/2410.16271/) (2024)
- [AlignMiF: Geometry-Aligned Multimodal Implicit Field for LiDAR-Camera Joint Synthesis](https://mlsanity.com/paper/2402.17483/) (2024)
- [HO-Gaussian: Hybrid Optimization of 3D Gaussian Splatting for Urban Scenes](https://mlsanity.com/paper/2403.20032/) (2024)
- [NeRFs in Robotics: A Survey](https://mlsanity.com/paper/2405.01333/) (2024)
- [PuzzleAvatar: Assembling 3D Avatars from Personal Albums](https://mlsanity.com/paper/2405.14869/) (2024)

## Abstract (arXiv)

We present an algorithm for reconstructing the radiance field of a large-scale scene from a single casually captured video. The task poses two core challenges. First, most existing radiance field reconstruction approaches rely on accurate pre-estimated camera poses from Structure-from-Motion algorithms, which frequently fail on in-the-wild videos. Second, using a single, global radiance field with finite representational capacity does not scale to longer trajectories in an unbounded scene. For handling unknown poses, we jointly estimate the camera poses with radiance field in a progressive manner. We show that progressive optimization significantly improves the robustness of the reconstruction. For handling large unbounded scenes, we dynamically allocate new local radiance fields trained with frames within a temporal window. This further improves robustness (e.g., performs well even under moderate pose drifts) and allows us to scale to large scenes. Our extensive evaluation on the Tanks and Temples dataset and our collected outdoor dataset, Static Hikes, show that our approach compares favorably with the state-of-the-art.