mlsanityData to 14 Oct 2026

How to Train Neural Field Representations: A Comprehensive Study and Benchmark

arXiv 2312.10531 ↗First posted 16 Dec 2023cs.CVcited by 12

Not on the map. Why?

Abstract (arXiv)

Neural fields (NeFs) have recently emerged as a versatile method for modeling signals of various modalities, including images, shapes, and scenes. Subsequently, a number of works have explored the use of NeFs as representations for downstream tasks, e.g. classifying an image based on the parameters of a NeF that has been fit to it. However, the impact of the NeF hyperparameters on their quality as downstream representation is scarcely understood and remains largely unexplored. This is in part caused by the large amount of time required to fit datasets of neural fields. In this work, we propose a JAX-based library that leverages parallelization to enable fast optimization of large-scale NeF datasets, resulting in a significant speed-up. With this library, we perform a comprehensive study that investigates the effects of different hyperparameters on fitting NeFs for downstream tasks. In particular, we explore the use of a shared initialization, the effects of overtraining, and the expressiveness of the network architectures used. Our study provides valuable insights on how to train NeFs and offers guidance for optimizing their effectiveness in downstream applications. Finally, based on the proposed library and our analysis, we propose Neural Field Arena, a benchmark consisting of neural field variants of popular vision datasets, including MNIST, CIFAR, variants of ImageNet, and ShapeNetv2. Our library and the Neural Field Arena will be open-sourced to introduce standardized benchmarking and promote further research on neural fields.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 38 of its 52 references that are in the map. First posted on arXiv 16 Dec 2023.

Builds on

38 of 52 references.

References · 38

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  2. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  3. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  4. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  5. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,501
  6. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
  7. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,238
  8. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,200
  9. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  10. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,584
  11. 2017Deep Setscited by 1,421
  12. 2018On the Spectral Bias of Neural Networkscited by 1,341
26 more
  1. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,198
  2. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 941
  3. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  4. 2021Neural Fields in Visual Computing and Beyondcited by 540
  5. 2020Principal Neighbourhood Aggregation for Graph Netscited by 478
  6. 2022Git Re-Basin: Merging Models modulo Permutation Symmetriescited by 471
  7. 2019Model Fusion via Optimal Transportcited by 219
  8. 2022From data to functa: Your data point is a function and you can treat it like onecited by 187
  9. 2023HyperDiffusion: Generating Implicit Neural Fields with Weight-Space Diffusioncited by 152
  10. 2021Modulated Periodic Activations for Generalizable Local Functional Representationscited by 140
  11. 2020Predicting Neural Network Accuracy from Weightscited by 110
  12. 2022NAF: Neural Attenuation Fields for Sparse-View CBCT Reconstructioncited by 108
  13. 2023Equivariant Architectures for Learning in Deep Weight Spacescited by 107
  14. 2022Continuous PDE Dynamics Forecasting with Implicit Neural Representationscited by 73
  15. 2023Permutation Equivariant Neural Functionalscited by 70
  16. 2021ObjectFolder: A Dataset of Objects with Implicit Visual, Auditory, and Tactile Representationscited by 68
  17. 2021FlexConv: Continuous Kernel Convolutions with Differentiable Kernel Sizescited by 66
  18. 2020Classifying the classifier: dissecting the weight space of neural networkscited by 65
  19. 2019Weight-space symmetry in deep networks gives rise to permutation saddles, connected by equal-loss valleys across the loss landscapecited by 63
  20. 2021SAPE: Spatially-Adaptive Progressive Encoding for Neural Optimizationcited by 59
  21. 2023Spatial Functa: Scaling Functa to ImageNet Classification and Generationcited by 57
  22. 2022Signal Processing for Implicit Neural Representationscited by 46
  23. 2022Relational Attention: Generalizing Transformers for Graph-Structured Taskscited by 41
  24. 2022PINs: Progressive Implicit Networks for Multi-Scale Neural Representationscited by 19
  25. 2023Latent Field Discovery In Interacting Dynamical Systems With Neural Fieldscited by 7
  26. 2023Neural Modulation Fields for Conditional Cone Beam Neural Tomographycited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 7; 2026: 1.

12 papers.

  1. 2024Grounding Continuous Representations in Geometry: Equivariant Neural Fieldscited by 19
  2. 2024Deep Linear Probe Generators for Weight Space Learningcited by 17
  3. 2024Fast Training of Sinusoidal Neural Fields via Scaling Initializationcited by 11
  4. 2024Latent-INR: A Flexible Framework for Implicit Representations of Videos with Discriminative Semanticscited by 9
  5. 2025Optimizing Rank for High-Fidelity Implicit Neural Representationscited by 8
  6. 2025End-to-End Implicit Neural Representations for Classificationcited by 7
  7. 2025ARC: Anchored Representation Clouds for High-Resolution INR Classificationcited by 3
  8. 2025VidFuncta: Towards Generalizable Neural Representations for Ultrasound Videoscited by 2
  9. 2025Beyond Pixels: Medical Image Quality Assessment with Implicit Neural Representationscited by 1
  10. 2025Geometric Neural Process Fieldscited by 0
  11. 2025On the Internal Representations of Graph Metanetworkscited by 0
  12. 2026Is Class Signal Clustered or Routed in Task-Induced Implicit Neural Representation Weight Spaces?cited by 0