mlsanityData to 14 Oct 2026

Nonparametric Teaching of Implicit Neural Representations

arXiv 2405.10531 ↗First posted 17 May 2024cs.LG cs.CVcited by 15

Not on the map. Why?

Abstract (arXiv)

We investigate the learning of implicit neural representation (INR) using an overparameterized multilayer perceptron (MLP) via a novel nonparametric teaching perspective. The latter offers an efficient example selection framework for teaching nonparametrically defined (viz. non-closed-form) target functions, such as image functions defined by 2D grids of pixels. To address the costly training of INRs, we propose a paradigm called Implicit Neural Teaching (INT) that treats INR learning as a nonparametric teaching problem, where the given signal being fitted serves as the target function. The teacher then selects signal fragments for iterative training of the MLP to achieve fast convergence. By establishing a connection between MLP evolution through parameter-based gradient descent and that of function evolution through functional gradient descent in nonparametric teaching, we show for the first time that teaching an overparameterized MLP is consistent with teaching a nonparametric learner. This new discovery readily permits a convenient drop-in of nonparametric teaching algorithms to broadly enhance INR training efficiency, demonstrating 30%+ training time savings across various input modalities.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 54 of its 80 references that are in the map. First posted on arXiv 17 May 2024.

Builds on

54 of 80 references.

References · 54

  1. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  2. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  3. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  4. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  5. 2021Convex optimizationcited by 2,515
  6. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,238
  7. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,200
  8. 2018On the Spectral Bias of Neural Networkscited by 1,341
  9. 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,232
  10. 2016An overview of gradient descent optimization algorithmscited by 1,033
  11. 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 738
  12. 2020Implicit Geometric Regularization for Learning Shapescited by 710
38 more
  1. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 673
  2. 2016Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithmcited by 507
  3. 2019SAL: Sign Agnostic Learning of Shapes from Raw Datacited by 390
  4. 2017Automated Curriculum Learning for Neural Networkscited by 257
  5. 2020Learned Initializations for Optimizing Coordinate-Based Neural Representationscited by 242
  6. 2022Prioritized Training on Points that are Learnable, Worth Learning, and Not Yet Learntcited by 236
  7. 2021COIN: COmpression with Implicit Neural representationscited by 233
  8. 2015Coordinate Descent Algorithmscited by 219
  9. 2020MetaSDF: Meta-learning Signed Distance Functionscited by 214
  10. 2022From data to functa: Your data point is a function and you can treat it like onecited by 187
  11. 2019On the Inductive Bias of Neural Tangent Kernelscited by 177
  12. 2021Implicit Neural Representations for Image Compressioncited by 162
  13. 2017Stein Variational Gradient Descent as Gradient Flowcited by 152
  14. 2021BACON: Band-limited Coordinate Networks for Multiscale Scene Representationcited by 143
  15. 2019Maximum Mean Discrepancy Gradient Flowcited by 127
  16. 2015Online Batch Selection for Faster Training of Neural Networkscited by 122
  17. 2021A Structured Dictionary Perspective on Implicit Neural Representationscited by 100
  18. 2023Implicit Neural Representation in Medical Imaging: A Comparative Surveycited by 90
  19. 2020Policy Teaching via Environment Poisoning: Training-time Adversarial Attacks against Reinforcement Learningcited by 81
  20. 2020On the Similarity between the Laplace and Neural Tangent Kernelscited by 76
  21. 2020Deep Neural Tangent Kernel and Laplace Kernel Have the Same RKHScited by 74
  22. 2019Policy Poisoning in Batch Reinforcement Learning and Controlcited by 70
  23. 2018An Overview of Machine Teachingcited by 63
  24. 2019Convergence of Adversarial Training in Overparametrized Neural Networkscited by 58
  25. 2017Iterative Machine Teachingcited by 41
  26. 2021NeRF in detail: Learning to sample for view synthesiscited by 35
  27. 2014Near-Optimally Teaching the Crowd to Classifycited by 32
  28. 2022Generalised Implicit Neural Representationscited by 31
  29. 2023Disorder-invariant Implicit Neural Representationcited by 27
  30. 2023Modality-Agnostic Variational Compression of Implicit Neural Representationscited by 22
  31. 2021A Multi-Implicit Neural Representation for Fontscited by 21
  32. 2017Towards Black-box Iterative Machine Teachingcited by 20
  33. 2019Training Neural Networks as Learning Data-adaptive Kernels: Provable Representation and Approximation Benefitscited by 13
  34. 2023Learning Large-scale Neural Fields via Context Pruned Meta-Learningcited by 13
  35. 2023Regularize implicit neural representation by itselfcited by 13
  36. 2018Unlearn What You Have Learned: Adaptive Crowd Teaching with Exponentially Decayed Memory Learnerscited by 11
  37. 2023Learning Spatially Collaged Fourier Bases for Implicit Neural Representationcited by 11
  38. 2020Sinkhorn Barycenter via Functional Gradient Descentcited by 9

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 7; 2026: 5.

15 papers.

Sort:

  1. 2024Imaging Interiors: An Implicit Solution to Electromagnetic Inverse Scattering Problemscited by 8
  2. 2024Inductive Gradient Adjustment For Spectral Bias In Implicit Neural Representationscited by 8
  3. 2025Optimizing Rank for High-Fidelity Implicit Neural Representationscited by 8
  4. 2024EVOS: Efficient Implicit Neural Training via EVOlutionary Selectorcited by 7
  5. 2025Nonparametric Teaching for Graph Property Learnerscited by 5
  6. 2025Continuous Representation Methods, Theories, and Applications: An Overview and Perspectivescited by 4
  7. 2025NTK-Guided Implicit Neural Teachingcited by 2
  8. 2025MINR: Efficient Implicit Neural Representations for Multi-Image Encodingcited by 1
  9. 2025Distribution-Aware Hadamard Quantization for Hardware-Efficient Implicit Neural Representationscited by 1
  10. 2025Enhancing Robustness of Implicit Neural Representations Against Weight Perturbationscited by 1
  11. 2026Nonparametric Teaching of Attention Learnerscited by 1
  12. 2026Model Evolution Under Zeroth-Order Optimization: A Neural Tangent Kernel Perspectivecited by 0
3 more
  1. 2026PatchINR: Patch-Based Implicit Neural Representations for Efficient and Scalable Inferencecited by 0
  2. 2026Efficient Neural Field Learning via Adaptive Coverage and Focused Samplingcited by 0
  3. 2026Refined Grid Sampling for Efficient Implicit Neural Representations of Large-Scale Scientific Datacited by 0