# Disentanglement with Biological Constraints: A Theory of Functional Cell Types

arXiv: https://arxiv.org/abs/2210.01768 · HTML: https://mlsanity.com/paper/2210.01768/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 96 references that are in the map.
- First posted on arXiv 30 Sep 2022.

## Builds on (most cited first, 25 of 25)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning](https://mlsanity.com/paper/2105.04906/) (2021)
- [Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations](https://mlsanity.com/paper/1811.12359/) (2018)
- [Network Dissection: Quantifying Interpretability of Deep Visual Representations](https://mlsanity.com/paper/1704.05796/) (2017)
- [Disentangling by Factorising](https://mlsanity.com/paper/1802.05983/) (2018)
- [Variational Autoencoders and Nonlinear ICA: A Unifying Framework](https://mlsanity.com/paper/1907.04809/) (2019)
- [Variational Inference of Disentangled Latent Concepts from Unlabeled Observations](https://mlsanity.com/paper/1711.00848/) (2017)
- [Isolating Sources of Disentanglement in Variational Autoencoders](https://mlsanity.com/paper/1802.04942/) (2018)
- [DARLA: Improving Zero-Shot Transfer in Reinforcement Learning](https://mlsanity.com/paper/1707.08475/) (2017)
- [Learning Deep Disentangled Embeddings with the F-Statistic Loss](https://mlsanity.com/paper/1802.05312/) (2018)
- [Variational Autoencoders Pursue PCA Directions (by Accident)](https://mlsanity.com/paper/1812.06775/) (2018)
- [Relating transformers to models and neural representations of the hippocampal formation](https://mlsanity.com/paper/2112.04035/) (2021)
- [Visual Representation Learning Does Not Generalize Strongly Within the Same Domain](https://mlsanity.com/paper/2107.08221/) (2021)
- [Emergence of grid-like representations by training recurrent neural networks to perform spatial localization](https://mlsanity.com/paper/1803.07770/) (2018)
- [Loss Landscapes of Regularized Linear Autoencoders](https://mlsanity.com/paper/1901.08168/) (2019)
- [Deep Learning Models of the Retinal Response to Natural Scenes](https://mlsanity.com/paper/1702.01825/) (2017)
- [Unsupervised deep learning identifies semantic disentanglement in single inferotemporal neurons](https://mlsanity.com/paper/2006.14304/) (2020)
- [A Hebbian/Anti-Hebbian Neural Network for Linear Subspace Learning: A Derivation from Multidimensional Scaling of Streaming Data](https://mlsanity.com/paper/1503.00669/) (2015)
- [Blind nonnegative source separation using biological neural networks](https://mlsanity.com/paper/1706.00382/) (2017)
- [On Path Integration of Grid Cells: Group Representation and Isotropic Scaling](https://mlsanity.com/paper/2006.10259/) (2020)
- [Biologically plausible single-layer networks for nonnegative independent component analysis](https://mlsanity.com/paper/2010.12632/) (2020)
- [Biologically-Plausible Determinant Maximization Neural Networks for Blind Separation of Correlated Sources](https://mlsanity.com/paper/2209.12894/) (2022)
- [Demystifying Inductive Biases for β-VAE Based Architectures](https://mlsanity.com/paper/2102.06822/) (2021)
- [Constellation: Learning relational abstractions over objects for compositional imagination](https://mlsanity.com/paper/2107.11153/) (2021)

## Built on by (most cited first, 30 of 30)

- [Mechanistic Interpretability for AI Safety -- A Review](https://mlsanity.com/paper/2404.14082/) (2024)
- [Disentanglement via Latent Quantization](https://mlsanity.com/paper/2305.18378/) (2023)
- [When does compositional structure yield compositional generalization? A kernel theory](https://mlsanity.com/paper/2405.16391/) (2024)
- [Residual Stream Analysis with Multi-Layer SAEs](https://mlsanity.com/paper/2409.04185/) (2024)
- [Tripod: Three Complementary Inductive Biases for Disentangled Representation Learning](https://mlsanity.com/paper/2404.10282/) (2024)
- [Does Data Scaling Lead to Visual Compositional Generalization?](https://mlsanity.com/paper/2507.07102/) (2025)
- [Flow Factorized Representation Learning](https://mlsanity.com/paper/2309.13167/) (2023)
- [Poisson Variational Autoencoder](https://mlsanity.com/paper/2405.14473/) (2024)
- [Statistical and structural identifiability in representation learning](https://mlsanity.com/paper/2603.11970/) (2026)
- [Probing Biological and Artificial Neural Networks with Task-dependent Neural Manifolds](https://mlsanity.com/paper/2312.14285/) (2023)
- [Disentangling Representations through Multi-task Learning](https://mlsanity.com/paper/2407.11249/) (2024)
- [Range, not Independence, Drives Modularity in Biologically Inspired Representations](https://mlsanity.com/paper/2410.06232/) (2024)
- [Zero-Shot Generalization of Vision-Based RL Without Data Augmentation](https://mlsanity.com/paper/2410.07441/) (2024)
- [A mathematical theory for understanding when abstract representations emerge in neural networks](https://mlsanity.com/paper/2510.09816/) (2025)
- [The Lie-Group Bayesian Learning Rule](https://mlsanity.com/paper/2303.04397/) (2023)
- [Correlative Information Maximization: A Biologically Plausible Approach to Supervised Deep Neural Networks without Weight Symmetry](https://mlsanity.com/paper/2306.04810/) (2023)
- [Zero-Shot Visual Generalization in Robot Manipulation](https://mlsanity.com/paper/2505.11719/) (2025)
- [A Controllable Appearance Representation for Flexible Transfer and Editing](https://mlsanity.com/paper/2504.15028/) (2025)
- [Revisiting Disentanglement in Downstream Tasks: A Study on Its Necessity for Abstract Visual Reasoning](https://mlsanity.com/paper/2403.00352/) (2024)
- [Disentanglement with Factor Quantized Variational Autoencoders](https://mlsanity.com/paper/2409.14851/) (2024)
- [Is Generation Required for Data-Efficient Perception?](https://mlsanity.com/paper/2512.08854/) (2025)
- [Stimulus symmetries can confound representational similarity analyses](https://mlsanity.com/paper/2605.21324/) (2026)
- [Disentangled Representations for Causal Cognition](https://mlsanity.com/paper/2407.00744/) (2024)
- [Do Mice Grok? Glimpses of Hidden Progress During Overtraining in Sensory Cortex](https://mlsanity.com/paper/2411.03541/) (2024)
- [Disentanglement and Compositionality of Letter Identity and Letter Position in Variational Auto-Encoder Vision Models](https://mlsanity.com/paper/2412.10446/) (2024)
- [Vector Quantization in the Brain: Grid-like Codes in World Models](https://mlsanity.com/paper/2510.16039/) (2025)
- [MapFormer: Self-Supervised Learning of Cognitive Maps with Input-Dependent Positional Embeddings](https://mlsanity.com/paper/2511.19279/) (2025)
- [Modular connectivity in neural networks emerges from Poisson noise-motivated regularisation, and promotes robustness and compositional generalisation](https://mlsanity.com/paper/2512.13707/) (2025)
- [Metabolic cost of information processing in Poisson variational autoencoders](https://mlsanity.com/paper/2602.13421/) (2026)
- [The Computational Value of Sensory-Aligned Receptive Fields Depends on Neuronal Expressivity](https://mlsanity.com/paper/2609.26940/) (2026)

## Abstract (arXiv)

Neurons in the brain are often finely tuned for specific task variables. Moreover, such disentangled representations are highly sought after in machine learning. Here we mathematically prove that simple biological constraints on neurons, namely nonnegativity and energy efficiency in both activity and weights, promote such sought after disentangled representations by enforcing neurons to become selective for single factors of task variation. We demonstrate these constraints lead to disentanglement in a variety of tasks and architectures, including variational autoencoders. We also use this theory to explain why the brain partitions its cells into distinct cell types such as grid and object-vector cells, and also explain when the brain instead entangles representations in response to entangled task factors. Overall, this work provides a mathematical understanding of why single neurons in the brain often represent single human-interpretable factors, and steps towards an understanding task structure shapes the structure of brain representation.