# Escaping Spectral Bias without Backpropagation: Fast Implicit Neural Representations with Extreme Learning Machines

arXiv: https://arxiv.org/abs/2602.07603 · HTML: https://mlsanity.com/paper/2602.07603/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 46 references that are in the map.
- First posted on arXiv 7 Feb 2026.

## Builds on (most cited first, 28 of 28)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [COIN: COmpression with Implicit Neural representations](https://mlsanity.com/paper/2103.03123/) (2021)
- [WIRE: Wavelet Implicit Neural Representations](https://mlsanity.com/paper/2301.05187/) (2023)
- [From data to functa: Your data point is a function and you can treat it like one](https://mlsanity.com/paper/2201.12204/) (2022)
- [Generalization Properties of Learning with Random Features](https://mlsanity.com/paper/1602.04474/) (2016)
- [Finite Basis Physics-Informed Neural Networks (FBPINNs): a scalable domain decomposition approach for solving differential equations](https://mlsanity.com/paper/2107.07871/) (2021)
- [Beyond Periodicity: Towards a Unifying Framework for Activations in Coordinate-MLPs](https://mlsanity.com/paper/2111.15135/) (2021)
- [A Structured Dictionary Perspective on Implicit Neural Representations](https://mlsanity.com/paper/2112.01917/) (2021)
- [Implicit Neural Representation in Medical Imaging: A Comparative Survey](https://mlsanity.com/paper/2307.16142/) (2023)
- [InverseBench: Benchmarking Plug-and-Play Diffusion Priors for Inverse Problems in Physical Sciences](https://mlsanity.com/paper/2503.11043/) (2025)
- [Separable Physics-Informed Neural Networks](https://mlsanity.com/paper/2306.15969/) (2023)
- [Deep Learning on Implicit Neural Representations of Shapes](https://mlsanity.com/paper/2302.05438/) (2023)
- [Spatial Functa: Scaling Functa to ImageNet Classification and Generation](https://mlsanity.com/paper/2302.03130/) (2023)
- [Where Do We Stand with Implicit Neural Representations? A Technical and Performance Survey](https://mlsanity.com/paper/2411.03688/) (2024)
- [On the Representation of Solutions to Elliptic PDEs in Barron Spaces](https://mlsanity.com/paper/2106.07539/) (2021)
- [Partition of unity networks: deep hp-approximation](https://mlsanity.com/paper/2101.11256/) (2021)
- [Neural Network Approximations of PDEs Beyond Linearity: A Representational Perspective](https://mlsanity.com/paper/2210.12101/) (2022)
- [Implicit-Zoo: A Large-Scale Dataset of Neural Implicit Functions for 2D Images and 3D Scenes](https://mlsanity.com/paper/2406.17438/) (2024)
- [Random Fourier Features for Asymmetric Kernels](https://mlsanity.com/paper/2209.08461/) (2022)
- [PDEfuncta: Spectrally-Aware Neural Representation for PDE Solution Modeling](https://mlsanity.com/paper/2506.12790/) (2025)
- [Fourier-Modulated Implicit Neural Representation for Multispectral Satellite Image Compression](https://mlsanity.com/paper/2506.01234/) (2025)
- [Neural Functions for Learning Periodic Signal](https://mlsanity.com/paper/2506.09526/) (2025)

## Abstract (arXiv)

Training implicit neural representations (INRs) to capture fine-scale details typically relies on iterative backpropagation and is often hindered by spectral bias when the target exhibits highly non-uniform frequency content. We propose ELM-INR, a backpropagation-free INR that decomposes the domain into overlapping subdomains and fits each local problem using an Extreme Learning Machine (ELM) in closed form, replacing iterative optimization with stable linear least-squares solutions. This design yields fast and numerically robust reconstruction by combining local predictors through a partition of unity. To understand where approximation becomes difficult under fixed local capacity, we analyze the method from a spectral Barron norm perspective, which reveals that global reconstruction error is dominated by regions with high spectral complexity. Building on this insight, we introduce BEAM, an adaptive mesh refinement strategy that balances spectral complexity across subdomains to improve reconstruction quality in capacity-constrained regimes.