# A Measure of the Complexity of Neural Representations based on Partial Information Decomposition

arXiv: https://arxiv.org/abs/2209.10438 · HTML: https://mlsanity.com/paper/2209.10438/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 57 references that are in the map.
- First posted on arXiv 21 Sep 2022.

## Builds on (most cited first, 23 of 23)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Deep Learning and the Information Bottleneck Principle](https://mlsanity.com/paper/1503.02406/) (2015)
- [Disentangling by Factorising](https://mlsanity.com/paper/1802.05983/) (2018)
- [Fantastic Generalization Measures and Where to Find Them](https://mlsanity.com/paper/1912.02178/) (2019)
- [Opening the Black Box of Deep Neural Networks via Information](https://mlsanity.com/paper/1703.00810/) (2017)
- [Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations](https://mlsanity.com/paper/1609.07061/) (2016)
- [Deep Learning with Limited Numerical Precision](https://mlsanity.com/paper/1502.02551/) (2015)
- [Insights on representational similarity in neural networks with canonical correlation](https://mlsanity.com/paper/1806.05759/) (2018)
- [Intrinsic dimension of data representations in deep neural networks](https://mlsanity.com/paper/1905.12784/) (2019)
- [Convergent Learning: Do different neural networks learn the same representations?](https://mlsanity.com/paper/1511.07543/) (2015)
- [Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models](https://mlsanity.com/paper/1708.08296/) (2017)
- [A Theory of Usable Information Under Computational Constraints](https://mlsanity.com/paper/2002.10689/) (2020)
- [Scikit-dimension: a Python package for intrinsic dimension estimation](https://mlsanity.com/paper/2109.02596/) (2021)
- [Cold Case: The Lost MNIST Digits](https://mlsanity.com/paper/1905.10498/) (2019)
- [On Information Plane Analyses of Neural Network Classifiers -- A Review](https://mlsanity.com/paper/2003.09671/) (2020)
- [Understanding Convolutional Neural Networks with Information Theory: An Initial Exploration](https://mlsanity.com/paper/1804.06537/) (2018)
- [Bits and Pieces: Understanding Information Decomposition from Part-whole Relationships and Formal Logic](https://mlsanity.com/paper/2008.09535/) (2020)
- [Measuring multivariate redundant information with pointwise common change in surprisal](https://mlsanity.com/paper/1602.05063/) (2016)
- [Disentanglement Analysis with Partial Information Decomposition](https://mlsanity.com/paper/2108.13753/) (2021)
- [Usable Information and Evolution of Optimal Representations During Training](https://mlsanity.com/paper/2010.02459/) (2020)

## Built on by (most cited first, 11 of 11)

- [DiffusionPID: Interpreting Diffusion via Partial Information Decomposition](https://mlsanity.com/paper/2406.05191/) (2024)
- [Synergistic information supports modality integration and flexible learning in neural networks solving multiple tasks](https://mlsanity.com/paper/2210.02996/) (2022)
- [A General Framework for Interpretable Neural Learning based on Local Information-Theoretic Goal Functions](https://mlsanity.com/paper/2306.02149/) (2023)
- [A Brain-like Synergistic Core in LLMs Drives Behaviour and Learning](https://mlsanity.com/paper/2601.06851/) (2026)
- [A Comprehensive Information-Decomposition Analysis of Large Vision-Language Models](https://mlsanity.com/paper/2603.29676/) (2026)
- [What should a neuron aim for? Designing local objective functions based on information theory](https://mlsanity.com/paper/2412.02482/) (2024)
- [Broadcast Channel Cooperative Gain: An Operational Interpretation of Partial Information Decomposition](https://mlsanity.com/paper/2502.10878/) (2025)
- [Surveying the space of descriptions of a composite system with machine learning](https://mlsanity.com/paper/2411.18579/) (2024)
- [Sensory robustness through top-down feedback and neural stochasticity in recurrent vision models](https://mlsanity.com/paper/2508.07115/) (2025)
- [Redundancy Maximization as a Principle of Associative Memory Learning in Hopfield Networks](https://mlsanity.com/paper/2511.02584/) (2025)
- [Gaussian BROJA Partial Information Decomposition in Closed Form](https://mlsanity.com/paper/2610.13464/) (2026)

## Abstract (arXiv)

In neural networks, task-relevant information is represented jointly by groups of neurons. However, the specific way in which this mutual information about the classification label is distributed among the individual neurons is not well understood: While parts of it may only be obtainable from specific single neurons, other parts are carried redundantly or synergistically by multiple neurons. We show how Partial Information Decomposition (PID), a recent extension of information theory, can disentangle these different contributions. From this, we introduce the measure of "Representational Complexity", which quantifies the difficulty of accessing information spread across multiple neurons. We show how this complexity is directly computable for smaller layers. For larger layers, we propose subsampling and coarse-graining procedures and prove corresponding bounds on the latter. Empirically, for quantized deep neural networks solving the MNIST and CIFAR10 tasks, we observe that representational complexity decreases both through successive hidden layers and over training, and compare the results to related measures. Overall, we propose representational complexity as a principled and interpretable summary statistic for analyzing the structure and evolution of neural representations and complex systems in general.