# A Large-Scale Outdoor Multi-modal Dataset and Benchmark for Novel View Synthesis and Implicit Scene Reconstruction

arXiv: https://arxiv.org/abs/2301.06782 · HTML: https://mlsanity.com/paper/2301.06782/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 44 references that are in the map.
- First posted on arXiv 17 Jan 2023.

## Builds on (most cited first, 30 of 30)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [Volume Rendering of Neural Implicit Surfaces](https://mlsanity.com/paper/2106.12052/) (2021)
- [NeRF++: Analyzing and Improving Neural Radiance Fields](https://mlsanity.com/paper/2010.07492/) (2020)
- [Depth-supervised NeRF: Fewer Views and Faster Training for Free](https://mlsanity.com/paper/2107.02791/) (2021)
- [Block-NeRF: Scalable Large Scene Neural View Synthesis](https://mlsanity.com/paper/2202.05263/) (2022)
- [UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction](https://mlsanity.com/paper/2104.10078/) (2021)
- [RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputs](https://mlsanity.com/paper/2112.00724/) (2021)
- [Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields](https://mlsanity.com/paper/2112.03907/) (2021)
- [BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networks](https://mlsanity.com/paper/1911.10127/) (2019)
- [Mega-NeRF: Scalable Construction of Large-Scale NeRFs for Virtual Fly-Throughs](https://mlsanity.com/paper/2112.10703/) (2021)
- [NIMA: Neural Image Assessment](https://mlsanity.com/paper/1709.05424/) (2017)
- [BungeeNeRF: Progressive Neural Radiance Field for Extreme Multi-scale Scene Rendering](https://mlsanity.com/paper/2112.05504/) (2021)
- [GeoNeRF: Generalizing NeRF with Geometry Priors](https://mlsanity.com/paper/2111.13539/) (2021)
- [How2: A Large-scale Dataset for Multimodal Language Understanding](https://mlsanity.com/paper/1811.00347/) (2018)
- [Neural 3D Scene Reconstruction with the Manhattan-world Assumption](https://mlsanity.com/paper/2205.02836/) (2022)
- [TransformerFusion: Monocular RGB Scene Reconstruction using Transformers](https://mlsanity.com/paper/2107.02191/) (2021)
- [H3D-Net: Few-Shot High-Fidelity 3D Head Reconstruction](https://mlsanity.com/paper/2107.12512/) (2021)
- [ViewFormer: NeRF-free Neural Rendering from Few Images Using Transformers](https://mlsanity.com/paper/2203.10157/) (2022)
- [CoNeRF: Controllable Neural Radiance Fields](https://mlsanity.com/paper/2112.01983/) (2021)
- [Recursive-NeRF: An Efficient and Dynamically Growing NeRF](https://mlsanity.com/paper/2105.09103/) (2021)
- [Coordinates Are NOT Lonely -- Codebook Prior Helps Implicit Neural 3D Representations](https://mlsanity.com/paper/2210.11170/) (2022)
- [Robust 3D reconstruction of dynamic scenes from single-photon lidar using Beta-divergences](https://mlsanity.com/paper/2004.09211/) (2020)

## Built on by (most cited first, 30 of 32)

- [3D Gaussian Splatting as Markov Chain Monte Carlo](https://mlsanity.com/paper/2404.09591/) (2024)
- [MatrixCity: A Large-scale City Dataset for City-scale Neural Rendering and Beyond](https://mlsanity.com/paper/2309.16553/) (2023)
- [State of the Art on Diffusion Models for Visual Computing](https://mlsanity.com/paper/2310.07204/) (2023)
- [DiVa-360: The Dynamic Visual Dataset for Immersive Neural Fields](https://mlsanity.com/paper/2307.16897/) (2023)
- [ChatCam: Empowering Camera Control through Conversational AI](https://mlsanity.com/paper/2409.17331/) (2024)
- [VET: Visual Error Tomography for Point Cloud Completion and High-Quality Neural Rendering](https://mlsanity.com/paper/2311.04634/) (2023)
- [NextBestPath: Efficient 3D Mapping of Unseen Environments](https://mlsanity.com/paper/2502.05378/) (2025)
- [RobustSplat: Decoupling Densification and Dynamics for Transient-Free 3DGS](https://mlsanity.com/paper/2506.02751/) (2025)
- [3DGabSplat: 3D Gabor Splatting for Frequency-adaptive Radiance Field Rendering](https://mlsanity.com/paper/2508.05343/) (2025)
- [Evaluating Alternatives to SFM Point Cloud Initialization for Gaussian Splatting](https://mlsanity.com/paper/2404.12547/) (2024)
- [Editing 3D Scenes via Text Prompts without Retraining](https://mlsanity.com/paper/2309.04917/) (2023)
- [Scene123: One Prompt to 3D Scene Generation via Video-Assisted and Consistency-Enhanced MAE](https://mlsanity.com/paper/2408.05477/) (2024)
- [Robust 3D Semantic Occupancy Prediction with Calibration-free Spatial Transformation](https://mlsanity.com/paper/2411.12177/) (2024)
- [Bundle Adjustment in the Eager Mode](https://mlsanity.com/paper/2409.12190/) (2024)
- [A Survey on 3D Reconstruction Techniques in Plant Phenotyping: From Classical Methods to Neural Radiance Fields (NeRF), 3D Gaussian Splatting (3DGS), and Beyond](https://mlsanity.com/paper/2505.00737/) (2025)
- [ConeGS: Error-Guided Densification Using Pixel Cones for Improved Reconstruction With Fewer Primitives](https://mlsanity.com/paper/2511.06810/) (2025)
- [PDF: Point Diffusion Implicit Function for Large-scale Scene Neural Representation](https://mlsanity.com/paper/2311.01773/) (2023)
- [RPBG: Towards Robust Neural Point-based Graphics in the Wild](https://mlsanity.com/paper/2405.05663/) (2024)
- [Aria-NeRF: Multimodal Egocentric View Synthesis](https://mlsanity.com/paper/2311.06455/) (2023)
- [SS3DM: Benchmarking Street-View Surface Reconstruction with a Synthetic 3D Mesh Dataset](https://mlsanity.com/paper/2410.21739/) (2024)
- [ClaraVid: A Holistic Scene Reconstruction Benchmark From Aerial Perspective With Delentropy-Based Complexity Profiling](https://mlsanity.com/paper/2503.17856/) (2025)
- [LightCity: An Urban Dataset for Outdoor Inverse Rendering and Reconstruction under Multi-illumination Conditions](https://mlsanity.com/paper/2602.01118/) (2026)
- [Aerial Lifting: Neural Urban Semantic and Building Instance Lifting from Aerial Imagery](https://mlsanity.com/paper/2403.11812/) (2024)
- [Unconstrained Large-scale 3D Reconstruction and Rendering across Altitudes](https://mlsanity.com/paper/2505.00734/) (2025)
- [MTReD: 3D Reconstruction Dataset for Fly-over Videos of Maritime Domain](https://mlsanity.com/paper/2503.00853/) (2025)
- [Efficient multi-view training for 3D Gaussian Splatting](https://mlsanity.com/paper/2506.12727/) (2025)
- [Do Flat Minima Improve Sparse Novel View Synthesis?](https://mlsanity.com/paper/2511.17918/) (2025)
- [Towards High-Fidelity Gaussian Splatting with Queried-Convolution Neural Networks](https://mlsanity.com/paper/2512.12898/) (2025)
- [A Step to Decouple Optimization in 3DGS](https://mlsanity.com/paper/2601.16736/) (2026)
- [TurboGS: Accelerating 3D Gaussian Splatting via Error-Guided Sparse Pixel Sampling and Optimization](https://mlsanity.com/paper/2606.15924/) (2026)

## Abstract (arXiv)

Neural Radiance Fields (NeRF) has achieved impressive results in single object scene reconstruction and novel view synthesis, which have been demonstrated on many single modality and single object focused indoor scene datasets like DTU, BMVS, and NeRF Synthetic.However, the study of NeRF on large-scale outdoor scene reconstruction is still limited, as there is no unified outdoor scene dataset for large-scale NeRF evaluation due to expensive data acquisition and calibration costs. In this paper, we propose a large-scale outdoor multi-modal dataset, OMMO dataset, containing complex land objects and scenes with calibrated images, point clouds and prompt annotations. Meanwhile, a new benchmark for several outdoor NeRF-based tasks is established, such as novel view synthesis, surface reconstruction, and multi-modal NeRF. To create the dataset, we capture and collect a large number of real fly-view videos and select high-quality and high-resolution clips from them. Then we design a quality review module to refine images, remove low-quality frames and fail-to-calibrate scenes through a learning-based automatic evaluation plus manual review. Finally, a number of volunteers are employed to add the text descriptions for each scene and key-frame to meet the potential multi-modal requirements in the future. Compared with existing NeRF datasets, our dataset contains abundant real-world urban and natural scenes with various scales, camera trajectories, and lighting conditions. Experiments show that our dataset can benchmark most state-of-the-art NeRF methods on different tasks. We will release the dataset and model weights very soon.