# A Normative and Biologically Plausible Algorithm for Independent Component Analysis

arXiv: https://arxiv.org/abs/2111.08858 · HTML: https://mlsanity.com/paper/2111.08858/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 129 references that are in the map.
- First posted on arXiv 17 Nov 2021.

## Builds on (most cited first, 23 of 23)

- [Variational Autoencoders and Nonlinear ICA: A Unifying Framework](https://mlsanity.com/paper/1907.04809/) (2019)
- [Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learning](https://mlsanity.com/paper/1805.08651/) (2018)
- [Dendritic cortical microcircuits approximate the backpropagation algorithm](https://mlsanity.com/paper/1810.11393/) (2018)
- [Towards deep learning with segregated dendrites](https://mlsanity.com/paper/1610.00161/) (2016)
- [Learning Independent Features with Adversarial Nets for Non-linear ICA](https://mlsanity.com/paper/1710.05050/) (2017)
- [A Hebbian/Anti-Hebbian Neural Network for Linear Subspace Learning: A Derivation from Multidimensional Scaling of Streaming Data](https://mlsanity.com/paper/1503.00669/) (2015)
- [Why do similarity matching objectives lead to Hebbian/anti-Hebbian networks?](https://mlsanity.com/paper/1703.07914/) (2017)
- [Neuroscience-inspired online unsupervised learning algorithms](https://mlsanity.com/paper/1908.01867/) (2019)
- [Drawing Inspiration from Biological Dendrites to Empower Artificial Neural Networks](https://mlsanity.com/paper/2106.07490/) (2021)
- [A Normative Theory of Adaptive Dimensionality Reduction in Neural Networks](https://mlsanity.com/paper/1511.09426/) (2015)
- [The brain as an efficient and robust adaptive learner](https://mlsanity.com/paper/1705.08031/) (2017)
- [Blind nonnegative source separation using biological neural networks](https://mlsanity.com/paper/1706.00382/) (2017)
- [A Hebbian/Anti-Hebbian Network Derived from Online Non-Negative Matrix Factorization Can Cluster and Discover Sparse Features](https://mlsanity.com/paper/1503.00680/) (2015)
- [A biologically plausible neural network for multi-channel Canonical Correlation Analysis](https://mlsanity.com/paper/2010.00525/) (2020)
- [A biologically plausible neural network for Slow Feature Analysis](https://mlsanity.com/paper/2010.12644/) (2020)
- [Learning to represent signals spike by spike](https://mlsanity.com/paper/1703.03777/) (2017)
- [A simple normative network approximates local non-Hebbian learning in the cortex](https://mlsanity.com/paper/2010.12660/) (2020)
- [Biologically Plausible Online Principal Component Analysis Without Recurrent Neural Dynamics](https://mlsanity.com/paper/1810.06966/) (2018)
- [A Spiking Neural Network with Local Learning Rules Derived From Nonnegative Similarity Matching](https://mlsanity.com/paper/1902.01429/) (2019)
- [Blind Bounded Source Separation Using Neural Networks with Local Learning Rules](https://mlsanity.com/paper/2004.05479/) (2020)
- [A Similarity-preserving Neural Network Trained on Transformed Images Recapitulates Salient Features of the Fly Motion Detection Circuit](https://mlsanity.com/paper/2102.05503/) (2021)
- [Multi-set Canonical Correlation Analysis simply explained](https://mlsanity.com/paper/1802.03759/) (2018)
- [A Neural Network with Local Learning Rules for Minor Subspace Analysis](https://mlsanity.com/paper/2102.05501/) (2021)

## Built on by (most cited first, 8 of 8)

- [Biologically-Plausible Determinant Maximization Neural Networks for Blind Separation of Correlated Sources](https://mlsanity.com/paper/2209.12894/) (2022)
- [Normative framework for deriving neural networks with multi-compartmental neurons and non-Hebbian plasticity](https://mlsanity.com/paper/2302.10051/) (2023)
- [Beyond accuracy: generalization properties of bio-plausible temporal credit assignment rules](https://mlsanity.com/paper/2206.00823/) (2022)
- [Correlative Information Maximization Based Biologically Plausible Neural Networks for Correlated Source Separation](https://mlsanity.com/paper/2210.04222/) (2022)
- [Duality Principle and Biologically Plausible Learning: Connecting the Representer Theorem and Hebbian Learning](https://mlsanity.com/paper/2309.16687/) (2023)
- [Unsupervised Learning in Echo State Networks for Input Reconstruction](https://mlsanity.com/paper/2501.11409/) (2025)
- [Normative Networks for Source Separation via Local Plasticity and Dendritic Computation](https://mlsanity.com/paper/2605.19965/) (2026)
- [Flexible Online Representation Learning Based on Similarity Matching](https://mlsanity.com/paper/2606.01546/) (2026)

## Abstract (arXiv)

The brain effortlessly solves blind source separation (BSS) problems, but the algorithm it uses remains elusive. In signal processing, linear BSS problems are often solved by Independent Component Analysis (ICA). To serve as a model of a biological circuit, the ICA neural network (NN) must satisfy at least the following requirements: 1. The algorithm must operate in the online setting where data samples are streamed one at a time, and the NN computes the sources on the fly without storing any significant fraction of the data in memory. 2. The synaptic weight update is local, i.e., it depends only on the biophysical variables present in the vicinity of a synapse. Here, we propose a novel objective function for ICA from which we derive a biologically plausible NN, including both the neural architecture and the synaptic learning rules. Interestingly, our algorithm relies on modulating synaptic plasticity by the total activity of the output neurons. In the brain, this could be accomplished by neuromodulators, extracellular calcium, local field potential, or nitric oxide.