# Correlative Information Maximization: A Biologically Plausible Approach to Supervised Deep Neural Networks without Weight Symmetry

arXiv: https://arxiv.org/abs/2306.04810 · HTML: https://mlsanity.com/paper/2306.04810/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 70 references that are in the map.
- First posted on arXiv 7 Jun 2023.

## Builds on (most cited first, 25 of 25)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagation](https://mlsanity.com/paper/1602.05179/) (2016)
- [Dendritic cortical microcircuits approximate the backpropagation algorithm](https://mlsanity.com/paper/1810.11393/) (2018)
- [Deep Learning without Weight Transport](https://mlsanity.com/paper/1904.05391/) (2019)
- [Towards deep learning with segregated dendrites](https://mlsanity.com/paper/1610.00161/) (2016)
- [Scaling Equilibrium Propagation to Deep ConvNets by Drastically Reducing its Gradient Estimator Bias](https://mlsanity.com/paper/2006.03824/) (2020)
- [How Important is Weight Symmetry in Backpropagation?](https://mlsanity.com/paper/1510.05067/) (2015)
- [Holomorphic Equilibrium Propagation Computes Exact Gradients Through Finite Size Oscillations](https://mlsanity.com/paper/2209.00530/) (2022)
- [Self-Supervised Learning with an Information Maximization Criterion](https://mlsanity.com/paper/2209.07999/) (2022)
- [Disentanglement with Biological Constraints: A Theory of Functional Cell Types](https://mlsanity.com/paper/2210.01768/) (2022)
- [Robust Volume Minimization-Based Matrix Factorization for Remote Sensing and Document Clustering](https://mlsanity.com/paper/1608.04290/) (2016)
- [Backpropagation at the Infinitesimal Inference Limit of Energy-Based Models: Unifying Predictive Coding, Equilibrium Propagation, and Contrastive Hebbian Learning](https://mlsanity.com/paper/2206.02629/) (2022)
- [Generalization of Equilibrium Propagation to Vector Field Dynamics](https://mlsanity.com/paper/1808.04873/) (2018)
- [Constrained Predictive Coding as a Biologically Plausible Model of the Cortical Hierarchy](https://mlsanity.com/paper/2210.15752/) (2022)
- [Deep learning with asymmetric connections and Hebbian updates](https://mlsanity.com/paper/1812.07965/) (2018)
- [Polytopic Matrix Factorization: Determinant Maximization Based Criterion and Identifiability](https://mlsanity.com/paper/2202.09638/) (2022)
- [Blind nonnegative source separation using biological neural networks](https://mlsanity.com/paper/1706.00382/) (2017)
- [Understanding Neural Networks with Logarithm Determinant Entropy Estimator](https://mlsanity.com/paper/2105.03705/) (2021)
- [A biologically plausible neural network for multi-channel Canonical Correlation Analysis](https://mlsanity.com/paper/2010.00525/) (2020)
- [An Information Maximization Based Blind Source Separation Approach for Dependent and Independent Sources](https://mlsanity.com/paper/2205.00794/) (2022)
- [Biologically-Plausible Determinant Maximization Neural Networks for Blind Separation of Correlated Sources](https://mlsanity.com/paper/2209.12894/) (2022)
- [Correlative Information Maximization Based Biologically Plausible Neural Networks for Correlated Source Separation](https://mlsanity.com/paper/2210.04222/) (2022)

## Built on by (most cited first, 3 of 3)

- [Bidirectional predictive coding](https://mlsanity.com/paper/2505.23415/) (2025)
- [Normative Networks for Source Separation via Local Plasticity and Dendritic Computation](https://mlsanity.com/paper/2605.19965/) (2026)
- [Score Broadcast and Decorrelation: A General Framework for Broadcast-Based Credit Assignment](https://mlsanity.com/paper/2605.30638/) (2026)

## Abstract (arXiv)

The backpropagation algorithm has experienced remarkable success in training large-scale artificial neural networks; however, its biological plausibility has been strongly criticized, and it remains an open question whether the brain employs supervised learning mechanisms akin to it. Here, we propose correlative information maximization between layer activations as an alternative normative approach to describe the signal propagation in biological neural networks in both forward and backward directions. This new framework addresses many concerns about the biological-plausibility of conventional artificial neural networks and the backpropagation algorithm. The coordinate descent-based optimization of the corresponding objective, combined with the mean square error loss function for fitting labeled supervision data, gives rise to a neural network structure that emulates a more biologically realistic network of multi-compartment pyramidal neurons with dendritic processing and lateral inhibitory neurons. Furthermore, our approach provides a natural resolution to the weight symmetry problem between forward and backward signal propagation paths, a significant critique against the plausibility of the conventional backpropagation algorithm. This is achieved by leveraging two alternative, yet equivalent forms of the correlative mutual information objective. These alternatives intrinsically lead to forward and backward prediction networks without weight symmetry issues, providing a compelling solution to this long-standing challenge.