# Aug-NeRF: Training Stronger Neural Radiance Fields with Triple-Level Physically-Grounded Augmentations

arXiv: https://arxiv.org/abs/2207.01164 · HTML: https://mlsanity.com/paper/2207.01164/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Builds on 70 of its 89 references that are in the map.
- First posted on arXiv 4 Jul 2022.

## Builds on (most cited first, 30 of 70)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [Volume Rendering of Neural Implicit Surfaces](https://mlsanity.com/paper/2106.12052/) (2021)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [PlenOctrees for Real-time Rendering of Neural Radiance Fields](https://mlsanity.com/paper/2103.14024/) (2021)
- [AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty](https://mlsanity.com/paper/1912.02781/) (2019)
- [Robustness May Be at Odds with Accuracy](https://mlsanity.com/paper/1805.12152/) (2018)
- [NeRF++: Analyzing and Improving Neural Radiance Fields](https://mlsanity.com/paper/2010.07492/) (2020)
- [PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization](https://mlsanity.com/paper/1905.05172/) (2019)
- [MVSNet: Depth Inference for Unstructured Multi-view Stereo](https://mlsanity.com/paper/1804.02505/) (2018)
- [Convolutional Occupancy Networks](https://mlsanity.com/paper/2003.04618/) (2020)
- [MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo](https://mlsanity.com/paper/2103.15595/) (2021)
- [IBRNet: Learning Multi-View Image-Based Rendering](https://mlsanity.com/paper/2102.13090/) (2021)
- [Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision](https://mlsanity.com/paper/1912.07372/) (2019)
- [KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPs](https://mlsanity.com/paper/2103.13744/) (2021)
- [UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction](https://mlsanity.com/paper/2104.10078/) (2021)
- [Exploring Generalization in Deep Learning](https://mlsanity.com/paper/1706.08947/) (2017)

## Built on by (most cited first, 30 of 46)

- [FSGS: Real-Time Few-shot View Synthesis using Gaussian Splatting](https://mlsanity.com/paper/2312.00451/) (2023)
- [NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360deg Views](https://mlsanity.com/paper/2211.16431/) (2022)
- [Is Attention All That NeRF Needs?](https://mlsanity.com/paper/2207.13298/) (2022)
- [Physics-Informed Machine Learning: A Survey on Problems, Methods and Applications](https://mlsanity.com/paper/2211.08064/) (2022)
- [DiffDreamer: Towards Consistent Unsupervised Single-view Scene Extrapolation with Conditional Diffusion Models](https://mlsanity.com/paper/2211.12131/) (2022)
- [3DRealCar: An In-the-wild RGB-D Car Dataset with 360-degree Views](https://mlsanity.com/paper/2406.04875/) (2024)
- [A survey of synthetic data augmentation methods in computer vision](https://mlsanity.com/paper/2403.10075/) (2024)
- [NeRFLiX: High-Quality Neural View Synthesis by Learning a Degradation-Driven Inter-viewpoint MiXer](https://mlsanity.com/paper/2303.06919/) (2023)
- [ScatterNeRF: Seeing Through Fog with Physically-Based Inverse Neural Rendering](https://mlsanity.com/paper/2305.02103/) (2023)
- [Physics-Informed Computer Vision: A Review and Perspectives](https://mlsanity.com/paper/2305.18035/) (2023)
- [360FusionNeRF: Panoramic Neural Radiance Fields with Joint Guidance](https://mlsanity.com/paper/2209.14265/) (2022)
- [Reconstructing 4D Spatial Intelligence: A Survey](https://mlsanity.com/paper/2507.21045/) (2025)
- [Benchmarking Robustness in Neural Radiance Fields](https://mlsanity.com/paper/2301.04075/) (2023)
- [SpectralNeRF: Physically Based Spectral Rendering with Neural Radiance Field](https://mlsanity.com/paper/2312.08692/) (2023)
- [Where and How: Mitigating Confusion in Neural Radiance Fields from Sparse Inputs](https://mlsanity.com/paper/2308.02908/) (2023)
- [GaussianStego: A Generalizable Stenography Pipeline for Generative 3D Gaussians Splatting](https://mlsanity.com/paper/2407.01301/) (2024)
- [D2GS: Depth-and-Density Guided Gaussian Splatting for Stable and Accurate Sparse-View Reconstruction](https://mlsanity.com/paper/2510.08566/) (2025)
- [NeRFool: Uncovering the Vulnerability of Generalizable Neural Radiance Fields against Adversarial Perturbations](https://mlsanity.com/paper/2306.06359/) (2023)
- [RS-NeRF: Neural Radiance Fields from Rolling Shutter Images](https://mlsanity.com/paper/2407.10267/) (2024)
- [SpectralGaussians: Semantic, spectral 3D Gaussian splatting for multi-spectral scene representation, visualization and analysis](https://mlsanity.com/paper/2408.06975/) (2024)
- [SPARS3R: Semantic Prior Alignment and Regularization for Sparse 3D Reconstruction](https://mlsanity.com/paper/2411.12592/) (2024)
- [SparSplat: Fast Multi-View Reconstruction with Generalizable 2D Gaussian Splatting](https://mlsanity.com/paper/2505.02175/) (2025)
- [VM-NeRF: Tackling Sparsity in NeRF with View Morphing](https://mlsanity.com/paper/2210.04214/) (2022)
- [Clean-NeRF: Reformulating NeRF to account for View-Dependent Observations](https://mlsanity.com/paper/2303.14707/) (2023)
- [C-NERF: Representing Scene Changes as Directional Consistency Difference-based NeRF](https://mlsanity.com/paper/2312.02751/) (2023)
- [SSNeRF: Sparse View Semi-supervised Neural Radiance Fields with Augmentation](https://mlsanity.com/paper/2408.09144/) (2024)
- [HiCoM: Hierarchical Coherent Motion for Streamable Dynamic Scene with 3D Gaussian Splatting](https://mlsanity.com/paper/2411.07541/) (2024)
- [PDF: Point Diffusion Implicit Function for Large-scale Scene Neural Representation](https://mlsanity.com/paper/2311.01773/) (2023)
- [MF-NeRF: Memory Efficient NeRF with Mixed-Feature Hash Table](https://mlsanity.com/paper/2304.12587/) (2023)
- [CombiNeRF: A Combination of Regularization Techniques for Few-Shot Neural Radiance Field View Synthesis](https://mlsanity.com/paper/2403.14412/) (2024)

## Abstract (arXiv)

Neural Radiance Field (NeRF) regresses a neural parameterized scene by differentially rendering multi-view images with ground-truth supervision. However, when interpolating novel views, NeRF often yields inconsistent and visually non-smooth geometric results, which we consider as a generalization gap between seen and unseen views. Recent advances in convolutional neural networks have demonstrated the promise of advanced robust data augmentations, either random or learned, in enhancing both in-distribution and out-of-distribution generalization. Inspired by that, we propose Augmented NeRF (Aug-NeRF), which for the first time brings the power of robust data augmentations into regularizing the NeRF training. Particularly, our proposal learns to seamlessly blend worst-case perturbations into three distinct levels of the NeRF pipeline with physical grounds, including (1) the input coordinates, to simulate imprecise camera parameters at image capture; (2) intermediate features, to smoothen the intrinsic feature manifold; and (3) pre-rendering output, to account for the potential degradation factors in the multi-view image supervision. Extensive results demonstrate that Aug-NeRF effectively boosts NeRF performance in both novel view synthesis (up to 1.5dB PSNR gain) and underlying geometry reconstruction. Furthermore, thanks to the implicit smooth prior injected by the triple-level augmentations, Aug-NeRF can even recover scenes from heavily corrupted images, a highly challenging setting untackled before. Our codes are available in https://github.com/VITA-Group/Aug-NeRF.