# Improving Accuracy and Efficiency of Implicit Neural Representations: Making SIREN a WINNER

arXiv: https://arxiv.org/abs/2509.12980 · HTML: https://mlsanity.com/paper/2509.12980/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 25 references that are in the map.
- First posted on arXiv 16 Sep 2025.

## Builds on (most cited first, 17 of 17)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [Image Quality Assessment: Unifying Structure and Texture Similarity](https://mlsanity.com/paper/2004.07728/) (2020)
- [SAL: Sign Agnostic Learning of Shapes from Raw Data](https://mlsanity.com/paper/1911.10414/) (2019)
- [Noise2Self: Blind Denoising by Self-Supervision](https://mlsanity.com/paper/1901.11365/) (2019)
- [WIRE: Wavelet Implicit Neural Representations](https://mlsanity.com/paper/2301.05187/) (2023)
- [Modulated Periodic Activations for Generalizable Local Functional Representations](https://mlsanity.com/paper/2104.03960/) (2021)
- [Beyond Periodicity: Towards a Unifying Framework for Activations in Coordinate-MLPs](https://mlsanity.com/paper/2111.15135/) (2021)
- [Generalizable Implicit Neural Representations via Instance Pattern Composers](https://mlsanity.com/paper/2211.13223/) (2022)
- [Tuning the Frequencies: Robust Training for Sinusoidal Neural Networks](https://mlsanity.com/paper/2407.21121/) (2024)
- [FINER++: Building a Family of Variable-periodic Functions for Activating Implicit Neural Representation](https://mlsanity.com/paper/2407.19434/) (2024)
- [Fast Training of Sinusoidal Neural Fields via Scaling Initialization](https://mlsanity.com/paper/2410.04779/) (2024)
- [ASMR: Activation-sharing Multi-resolution Coordinate Networks For Efficient Inference](https://mlsanity.com/paper/2405.12398/) (2024)
- [Spectral Bottleneck in Sinusoidal Representation Networks: Noise is All You Need](https://mlsanity.com/paper/2509.09719/) (2025)
- [VI3NR: Variance Informed Initialization for Implicit Neural Representations](https://mlsanity.com/paper/2504.19270/) (2025)

## Built on by (most cited first, 1 of 1)

- [Architecture Shapes Transfer Specificity in Implicit Neural Representations](https://mlsanity.com/paper/2606.06827/) (2026)

## Abstract (arXiv)

We identify and address a fundamental limitation of sinusoidal representation networks (SIRENs), a class of implicit neural representations. SIRENs Sitzmann et al. (2020), when not initialized appropriately, can struggle at fitting signals that fall outside their frequency support. In extreme cases, when the network's frequency support misaligns with the target spectrum, a 'spectral bottleneck' phenomenon is observed, where the model yields to a near-zero output and fails to recover even the frequency components that are within its representational capacity. To overcome this, we propose WINNER - Weight Initialization with Noise for Neural Representations. WINNER perturbs uniformly initialized weights of base SIREN with Gaussian noise - whose noise scales are adaptively determined by the spectral centroid of the target signal. Similar to random Fourier embeddings, this mitigates 'spectral bias' but without introducing additional trainable parameters. Our method achieves state-of-the-art audio fitting and significant gains in image and 3D shape fitting tasks over base SIREN. Beyond signal fitting, WINNER suggests new avenues in adaptive, target-aware initialization strategies for optimizing deep neural network training. For code and data visit cfdlabtechnion.github.io/siren_square/.