# Stochastic Preconditioning for Neural Field Optimization

arXiv: https://arxiv.org/abs/2505.20473 · HTML: https://mlsanity.com/paper/2505.20473/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 47 references that are in the map.
- First posted on arXiv 26 May 2025.

## Builds on (most cited first, 30 of 32)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [Plenoxels: Radiance Fields without Neural Networks](https://mlsanity.com/paper/2112.05131/) (2021)
- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [Volume Rendering of Neural Implicit Surfaces](https://mlsanity.com/paper/2106.12052/) (2021)
- [UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction](https://mlsanity.com/paper/2104.10078/) (2021)
- [Neuralangelo: High-Fidelity Neural Surface Reconstruction](https://mlsanity.com/paper/2306.03092/) (2023)
- [Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes](https://mlsanity.com/paper/2101.10994/) (2021)
- [SAL: Sign Agnostic Learning of Shapes from Raw Data](https://mlsanity.com/paper/1911.10414/) (2019)
- [FreeNeRF: Improving Few-shot Neural Rendering with Free Frequency Regularization](https://mlsanity.com/paper/2303.07418/) (2023)
- [BungeeNeRF: Progressive Neural Radiance Field for Extreme Multi-scale Scene Rendering](https://mlsanity.com/paper/2112.05504/) (2021)
- [ReLU Fields: The Little Non-linearity That Could](https://mlsanity.com/paper/2205.10824/) (2022)
- [SAPE: Spatially-Adaptive Progressive Encoding for Neural Optimization](https://mlsanity.com/paper/2104.09125/) (2021)
- [CamP: Camera Preconditioning for Neural Radiance Fields](https://mlsanity.com/paper/2308.10902/) (2023)
- [Signal Processing for Implicit Neural Representations](https://mlsanity.com/paper/2210.08772/) (2022)
- [Explicit Regularisation in Gaussian Noise Injections](https://mlsanity.com/paper/2007.07368/) (2020)
- [PET-NeuS: Positional Encoding Tri-Planes for Neural Surfaces](https://mlsanity.com/paper/2305.05594/) (2023)
- [Floaters No More: Radiance Field Gradient Scaling for Improved Near-Camera Training](https://mlsanity.com/paper/2305.02756/) (2023)
- [Compact Neural Graphics Primitives with Learned Hash Probing](https://mlsanity.com/paper/2312.17241/) (2023)
- [Sinusoidal Frequency Estimation by Gradient Descent](https://mlsanity.com/paper/2210.14476/) (2022)
- [Fast Training of Sinusoidal Neural Fields via Scaling Initialization](https://mlsanity.com/paper/2410.04779/) (2024)
- [BANF: Band-limited Neural Fields for Levels of Detail Reconstruction](https://mlsanity.com/paper/2404.13024/) (2024)
- [Neural Field Convolutions by Repeated Differentiation](https://mlsanity.com/paper/2304.01834/) (2023)
- [In Search of a Data Transformation That Accelerates Neural Field Training](https://mlsanity.com/paper/2311.17094/) (2023)
- [DiffCD: A Symmetric Differentiable Chamfer Distance for Neural Implicit Surface Fitting](https://mlsanity.com/paper/2407.17058/) (2024)
- [Coarse-to-fine Optimization for Speech Enhancement](https://mlsanity.com/paper/1908.08044/) (2019)

## Built on by (most cited first, 8 of 8)

- [Splat the Net: Radiance Fields with Splattable Neural Primitives](https://mlsanity.com/paper/2510.08491/) (2025)
- [Dark3R: Learning Structure from Motion in the Dark](https://mlsanity.com/paper/2603.05330/) (2026)
- [Off the Planckian Locus: Using 2D Chromaticity to Improve In-Camera Color](https://mlsanity.com/paper/2511.17133/) (2025)
- [Do Flat Minima Improve Sparse Novel View Synthesis?](https://mlsanity.com/paper/2511.17918/) (2025)
- [Blockwise Stabilized Adaptive Cubic Regularization with Subsolvers via Recurrence](https://mlsanity.com/paper/2608.22129/) (2026)
- [Spectral Prefiltering of Neural Fields](https://mlsanity.com/paper/2510.08394/) (2025)
- [Generative Shape Reconstruction with Geometry-Guided Langevin Dynamics](https://mlsanity.com/paper/2603.27016/) (2026)
- [High-Fidelity Surface Splatting-Based 3D Reconstruction from Multi-View Images](https://mlsanity.com/paper/2605.07254/) (2026)

## Abstract (arXiv)

Neural fields are a highly effective representation across visual computing. This work observes that fitting these fields is greatly improved by incorporating spatial stochasticity during training, and that this simple technique can replace or even outperform custom-designed hierarchies and frequency space constructions. The approach is formalized as implicitly operating on a blurred version of the field, evaluated in-expectation by sampling with Gaussian-distributed offsets. Querying the blurred field during optimization greatly improves convergence and robustness, akin to the role of preconditioners in numerical linear algebra. This implicit, sampling-based perspective fits naturally into the neural field paradigm, comes at no additional cost, and is extremely simple to implement. We describe the basic theory of this technique, including details such as handling boundary conditions, and extending to a spatially-varying blur. Experiments demonstrate this approach on representations including coordinate MLPs, neural hashgrids, triplanes, and more, across tasks including surface reconstruction and radiance fields. In settings where custom-designed hierarchies have already been developed, stochastic preconditioning nearly matches or improves their performance with a simple and unified approach; in settings without existing hierarchies it provides an immediate boost to quality and robustness.