# Nonparametric Teaching of Implicit Neural Representations

arXiv: https://arxiv.org/abs/2405.10531 · HTML: https://mlsanity.com/paper/2405.10531/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 54 of its 80 references that are in the map.
- First posted on arXiv 17 May 2024.

## Builds on (most cited first, 30 of 54)

- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [An overview of gradient descent optimization algorithms](https://mlsanity.com/paper/1609.04747/) (2016)
- [Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent](https://mlsanity.com/paper/1902.06720/) (2019)
- [Implicit Geometric Regularization for Learning Shapes](https://mlsanity.com/paper/2002.10099/) (2020)
- [Data-Efficient Image Recognition with Contrastive Predictive Coding](https://mlsanity.com/paper/1905.09272/) (2019)
- [Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm](https://mlsanity.com/paper/1608.04471/) (2016)
- [SAL: Sign Agnostic Learning of Shapes from Raw Data](https://mlsanity.com/paper/1911.10414/) (2019)
- [Automated Curriculum Learning for Neural Networks](https://mlsanity.com/paper/1704.03003/) (2017)
- [Learned Initializations for Optimizing Coordinate-Based Neural Representations](https://mlsanity.com/paper/2012.02189/) (2020)
- [Prioritized Training on Points that are Learnable, Worth Learning, and Not Yet Learnt](https://mlsanity.com/paper/2206.07137/) (2022)
- [COIN: COmpression with Implicit Neural representations](https://mlsanity.com/paper/2103.03123/) (2021)
- [Coordinate Descent Algorithms](https://mlsanity.com/paper/1502.04759/) (2015)
- [MetaSDF: Meta-learning Signed Distance Functions](https://mlsanity.com/paper/2006.09662/) (2020)
- [From data to functa: Your data point is a function and you can treat it like one](https://mlsanity.com/paper/2201.12204/) (2022)
- [On the Inductive Bias of Neural Tangent Kernels](https://mlsanity.com/paper/1905.12173/) (2019)
- [Implicit Neural Representations for Image Compression](https://mlsanity.com/paper/2112.04267/) (2021)
- [Stein Variational Gradient Descent as Gradient Flow](https://mlsanity.com/paper/1704.07520/) (2017)
- [BACON: Band-limited Coordinate Networks for Multiscale Scene Representation](https://mlsanity.com/paper/2112.04645/) (2021)
- [Maximum Mean Discrepancy Gradient Flow](https://mlsanity.com/paper/1906.04370/) (2019)
- [Online Batch Selection for Faster Training of Neural Networks](https://mlsanity.com/paper/1511.06343/) (2015)
- [A Structured Dictionary Perspective on Implicit Neural Representations](https://mlsanity.com/paper/2112.01917/) (2021)
- [Implicit Neural Representation in Medical Imaging: A Comparative Survey](https://mlsanity.com/paper/2307.16142/) (2023)

## Built on by (most cited first, 15 of 15)

- [Imaging Interiors: An Implicit Solution to Electromagnetic Inverse Scattering Problems](https://mlsanity.com/paper/2407.09352/) (2024)
- [Inductive Gradient Adjustment For Spectral Bias In Implicit Neural Representations](https://mlsanity.com/paper/2410.13271/) (2024)
- [Optimizing Rank for High-Fidelity Implicit Neural Representations](https://mlsanity.com/paper/2512.14366/) (2025)
- [EVOS: Efficient Implicit Neural Training via EVOlutionary Selector](https://mlsanity.com/paper/2412.10153/) (2024)
- [Nonparametric Teaching for Graph Property Learners](https://mlsanity.com/paper/2505.14170/) (2025)
- [Continuous Representation Methods, Theories, and Applications: An Overview and Perspectives](https://mlsanity.com/paper/2505.15222/) (2025)
- [NTK-Guided Implicit Neural Teaching](https://mlsanity.com/paper/2511.15487/) (2025)
- [MINR: Efficient Implicit Neural Representations for Multi-Image Encoding](https://mlsanity.com/paper/2508.13471/) (2025)
- [Distribution-Aware Hadamard Quantization for Hardware-Efficient Implicit Neural Representations](https://mlsanity.com/paper/2508.13478/) (2025)
- [Enhancing Robustness of Implicit Neural Representations Against Weight Perturbations](https://mlsanity.com/paper/2508.13481/) (2025)
- [Nonparametric Teaching of Attention Learners](https://mlsanity.com/paper/2602.20461/) (2026)
- [Model Evolution Under Zeroth-Order Optimization: A Neural Tangent Kernel Perspective](https://mlsanity.com/paper/2603.21169/) (2026)
- [PatchINR: Patch-Based Implicit Neural Representations for Efficient and Scalable Inference](https://mlsanity.com/paper/2606.25534/) (2026)
- [Efficient Neural Field Learning via Adaptive Coverage and Focused Sampling](https://mlsanity.com/paper/2610.02410/) (2026)
- [Refined Grid Sampling for Efficient Implicit Neural Representations of Large-Scale Scientific Data](https://mlsanity.com/paper/2610.14205/) (2026)

## Abstract (arXiv)

We investigate the learning of implicit neural representation (INR) using an overparameterized multilayer perceptron (MLP) via a novel nonparametric teaching perspective. The latter offers an efficient example selection framework for teaching nonparametrically defined (viz. non-closed-form) target functions, such as image functions defined by 2D grids of pixels. To address the costly training of INRs, we propose a paradigm called Implicit Neural Teaching (INT) that treats INR learning as a nonparametric teaching problem, where the given signal being fitted serves as the target function. The teacher then selects signal fragments for iterative training of the MLP to achieve fast convergence. By establishing a connection between MLP evolution through parameter-based gradient descent and that of function evolution through functional gradient descent in nonparametric teaching, we show for the first time that teaching an overparameterized MLP is consistent with teaching a nonparametric learner. This new discovery readily permits a convenient drop-in of nonparametric teaching algorithms to broadly enhance INR training efficiency, demonstrating 30%+ training time savings across various input modalities.