mlsanityData to 7 Oct 2026

Preconditioners for the Stochastic Training of Neural Fields

arXiv 2402.08784 ↗First posted 13 Feb 2024cs.CV cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

Neural fields encode continuous multidimensional signals as neural networks, enabling diverse applications in computer vision, robotics, and geometry. While Adam is effective for stochastic optimization, it often requires long training times. To address this, we explore alternative optimization techniques to accelerate training without sacrificing accuracy. Traditional second-order methods like L-BFGS are unsuitable for stochastic settings. We propose a theoretical framework for training neural fields with curvature-aware diagonal preconditioners, demonstrating their effectiveness across tasks such as image reconstruction, shape modeling, and Neural Radiance Fields (NeRF).

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 41 of its 61 references that are in the map. First posted on arXiv 13 Feb 2024.

Builds on

41 of 61 references.

References · 41

  1. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  2. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
  3. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  4. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,607
  5. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  6. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,459
  7. 2018On the Spectral Bias of Neural Networkscited by 1,321
  8. 2022TensoRF: Tensorial Radiance Fieldscited by 1,275
  9. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,194
  10. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,140
  11. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  12. 2016An overview of gradient descent optimization algorithmscited by 1,021
29 more
  1. 2021Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstructioncited by 961
  2. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  3. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  4. 2015Optimizing Neural Networks with Kronecker-factored Approximate Curvaturecited by 776
  5. 2019Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelinescited by 672
  6. 2021KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPscited by 672
  7. 2021BARF: Bundle-Adjusting Neural Radiance Fieldscited by 641
  8. 2021NICE-SLAM: Neural Implicit Scalable Encoding for SLAMcited by 586
  9. 2020Learning Continuous Image Representation with Local Implicit Image Functioncited by 550
  10. 2018Shampoo: Preconditioned Stochastic Tensor Optimizationcited by 413
  11. 2023WIRE: Wavelet Implicit Neural Representationscited by 220
  12. 2020ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learningcited by 201
  13. 2019NASA: Neural Articulated Shape Approximationcited by 149
  14. 2021BACON: Band-limited Coordinate Networks for Multiscale Scene Representationcited by 142
  15. 2020Adversarial Generation of Continuous Imagescited by 135
  16. 2021Beyond Periodicity: Towards a Unifying Framework for Activations in Coordinate-MLPscited by 134
  17. 20213D Neural Scene Representations for Visuomotor Controlcited by 110
  18. 2022GARF: Gaussian Activated Radiance Fields for High Fidelity Reconstruction and Pose Estimationcited by 86
  19. 2021SAPE: Spatially-Adaptive Progressive Encoding for Neural Optimizationcited by 59
  20. 2023Factor Fields: A Unified Framework for Neural Fields and Beyondcited by 58
  21. 2022SiNeRF: Sinusoidal Neural Radiance Fields for Joint Pose Estimation and Scene Reconstructioncited by 49
  22. 2015Preconditioned Stochastic Gradient Descentcited by 48
  23. 2021CoIL: Coordinate-based Internal Learning for Imaging Inverse Problemscited by 33
  24. 2024A Sampling Theory Perspective on Activations for Implicit Neural Representationscited by 23
  25. 2022Learning Implicit Priors for Motion Optimizationcited by 20
  26. 2022Trading Positional Complexity vs. Deepness in Coordinate Networkscited by 16
  27. 2023Curvature-Aware Training for Coordinate Networkscited by 8
  28. 2015Iteration Complexity Analysis of Multi-Block ADMM for a Family of Convex Minimization without Strong Convexitycited by 3
  29. 2022Neural Motion Fields: Encoding Grasp Trajectories as Implicit Value Functionscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 3; 2026: 1.

5 papers.

  1. 2025Optimizing Rank for High-Fidelity Implicit Neural Representationscited by 7
  2. 2025Continuous Representation Methods, Theories, and Applications: An Overview and Perspectivescited by 4
  3. 2026Blockwise Stabilized Adaptive Cubic Regularization with Subsolvers via Recurrencecited by 1
  4. 2024DR-BFR: Degradation Representation with Diffusion Models for Blind Face Restorationcited by 0
  5. 2025TetraSDF: Analytic Isosurface Extraction with Multi-resolution Tetrahedral Gridcited by 0