# Learnable Spectral Activations

arXiv: https://arxiv.org/abs/2610.07419 · HTML: https://mlsanity.com/paper/2610.07419/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 43 references that are in the map.
- First posted on arXiv 5 Oct 2026.

## Builds on (most cited first, 9 of 9)

- [EmerNeRF: Emergent Spatial-Temporal Scene Decomposition via Self-Supervision](https://mlsanity.com/paper/2311.02077/) (2023)
- [From data to functa: Your data point is a function and you can treat it like one](https://mlsanity.com/paper/2201.12204/) (2022)
- [PAC-NeRF: Physics Augmented Continuum Neural Radiance Fields for Geometry-Agnostic System Identification](https://mlsanity.com/paper/2303.05512/) (2023)
- [Solving Linear Inverse Problems Using the Prior Implicit in a Denoiser](https://mlsanity.com/paper/2007.13640/) (2020)
- [fKAN: Fractional Kolmogorov-Arnold Networks with trainable Jacobi basis functions](https://mlsanity.com/paper/2406.07456/) (2024)
- [A Sampling Theory Perspective on Activations for Implicit Neural Representations](https://mlsanity.com/paper/2402.05427/) (2024)
- [FINER++: Building a Family of Variable-periodic Functions for Activating Implicit Neural Representation](https://mlsanity.com/paper/2407.19434/) (2024)
- [Implicit Neural Representations with Fourier Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2409.09323/) (2024)
- [Adversarial Attacks in Weight-Space Classifiers](https://mlsanity.com/paper/2502.20314/) (2025)

## Abstract (arXiv)

Implicit neural representations (INRs) are shaped by the spectral structure induced by their input encodings and activation functions. Existing methods improve fitting primarily by modifying which frequencies are available to the network, through coordinate encodings or periodic nonlinearities. However, frequency access is not the only bottleneck: signals with localized or spatially varying structure require the network to efficiently compose frequencies into multi-harmonic internal responses. We introduce learnable spectral activations (LSA), which replace fixed neuron-level nonlinearities with a residual truncated Fourier series whose harmonic amplitudes are learned during training. LSA does not expand the asymptotic function class. Instead, it changes the factorization of the representation: linear weights select features while activation coefficients control spectral shaping, and the two are updated by separate gradients. Because the activation output is affine in the coefficients given fixed pre-activations, spectral tuning becomes a more direct subproblem compared to architectures where it is entangled with feature selection. Empirically, this factorization concentrates more target-signal energy in the leading eigenmodes of the neural tangent kernel, consistent with improved optimization behavior. Across audio, image, neural radiance field, and neural acoustic field tasks, LSA also improves reconstruction quality.