# Adaptive whitening in neural populations with gain-modulating interneurons

arXiv: https://arxiv.org/abs/2301.11955 · HTML: https://mlsanity.com/paper/2301.11955/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 48 references that are in the map.
- First posted on arXiv 27 Jan 2023.

## Builds on (most cited first, 11 of 11)

- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [Whitening for Self-Supervised Representation Learning](https://mlsanity.com/paper/2007.06346/) (2020)
- [On Feature Decorrelation in Self-Supervised Learning](https://mlsanity.com/paper/2105.00470/) (2021)
- [Nonlinear Transform Coding](https://mlsanity.com/paper/2007.03034/) (2020)
- [Optimal whitening and decorrelation](https://mlsanity.com/paper/1512.00809/) (2015)
- [Neuroscience-inspired online unsupervised learning algorithms](https://mlsanity.com/paper/1908.01867/) (2019)
- [Adaptive Denoising via GainTuning](https://mlsanity.com/paper/2107.12815/) (2021)
- [A Normative Theory of Adaptive Dimensionality Reduction in Neural Networks](https://mlsanity.com/paper/1511.09426/) (2015)
- [Interneurons accelerate learning dynamics in recurrent neural networks for statistical adaptation](https://mlsanity.com/paper/2209.10634/) (2022)
- [Multi-rate adaptive transform coding for video compression](https://mlsanity.com/paper/2210.14308/) (2022)

## Built on by (most cited first, 3 of 3)

- [Normative framework for deriving neural networks with multi-compartmental neurons and non-Hebbian plasticity](https://mlsanity.com/paper/2302.10051/) (2023)
- [Adaptive whitening with fast gain modulation and slow synaptic plasticity](https://mlsanity.com/paper/2308.13633/) (2023)
- [Dynamic gain neuromodulation attenuates the stability gap under joint training](https://mlsanity.com/paper/2507.14056/) (2025)

## Abstract (arXiv)

Statistical whitening transformations play a fundamental role in many computational systems, and may also play an important role in biological sensory systems. Existing neural circuit models of adaptive whitening operate by modifying synaptic interactions; however, such modifications would seem both too slow and insufficiently reversible. Motivated by the extensive neuroscience literature on gain modulation, we propose an alternative model that adaptively whitens its responses by modulating the gains of individual neurons. Starting from a novel whitening objective, we derive an online algorithm that whitens its outputs by adjusting the marginal variances of an overcomplete set of projections. We map the algorithm onto a recurrent neural network with fixed synaptic weights and gain-modulating interneurons. We demonstrate numerically that sign-constraining the gains improves robustness of the network to ill-conditioned inputs, and a generalization of the circuit achieves a form of local whitening in convolutional populations, such as those found throughout the visual or auditory systems.