# A Survey of Inductive Biases for Factorial Representation-Learning

arXiv: https://arxiv.org/abs/1612.05299 · HTML: https://mlsanity.com/paper/1612.05299/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Dec 2016.

## Built on by (most cited first, 23 of 23)

- [SODA: Bottleneck Diffusion Models for Representation Learning](https://mlsanity.com/paper/2311.17901/) (2023)
- [Investigating Explainability of Generative AI for Code through Scenario-based Design](https://mlsanity.com/paper/2202.04903/) (2022)
- [Disentanglement via Latent Quantization](https://mlsanity.com/paper/2305.18378/) (2023)
- [Controllable Data Generation by Deep Learning: A Review](https://mlsanity.com/paper/2207.09542/) (2022)
- [Evaluating the Interpretability of Generative Models by Interactive Reconstruction](https://mlsanity.com/paper/2102.01264/) (2021)
- [GANSlider: How Users Control Generative Models for Images using Multiple Sliders with and without Feedforward Information](https://mlsanity.com/paper/2202.00965/) (2022)
- [Benchmarks, Algorithms, and Metrics for Hierarchical Disentanglement](https://mlsanity.com/paper/2102.05185/) (2021)
- [VDSM: Unsupervised Video Disentanglement with State-Space Modeling and Deep Mixtures of Experts](https://mlsanity.com/paper/2103.07292/) (2021)
- [On Disentanglement in Gaussian Process Variational Autoencoders](https://mlsanity.com/paper/2102.05507/) (2021)
- [Disentangling Autoencoders (DAE)](https://mlsanity.com/paper/2202.09926/) (2022)
- [Are We Using Autoencoders in a Wrong Way?](https://mlsanity.com/paper/2309.01532/) (2023)
- [Invariance-based Multi-Clustering of Latent Space Embeddings for Equivariant Learning](https://mlsanity.com/paper/2107.11717/) (2021)
- [DAReN: A Collaborative Approach Towards Reasoning And Disentangling](https://mlsanity.com/paper/2109.13156/) (2021)
- [β-CapsNet: Learning Disentangled Representation for CapsNet by Information Bottleneck](https://mlsanity.com/paper/2209.05239/) (2022)
- [An Empirical Study on Clustering Pretrained Embeddings: Is Deep Strictly Better?](https://mlsanity.com/paper/2211.05183/) (2022)
- [LatentExplainer: Explaining Latent Representations in Deep Generative Models with Multimodal Large Language Models](https://mlsanity.com/paper/2406.14862/) (2024)
- [Transferring disentangled representations: bridging the gap between synthetic and real images](https://mlsanity.com/paper/2409.18017/) (2024)
- [IB-GAN: Disentangled Representation Learning with Information Bottleneck Generative Adversarial Networks](https://mlsanity.com/paper/2510.20165/) (2025)
- [Variational decomposition autoencoding improves disentanglement of latent representations](https://mlsanity.com/paper/2601.06844/) (2026)
- [Rethinking Disentanglement under Dependent Factors of Variation](https://mlsanity.com/paper/2408.07016/) (2024)
- [Disentangled representations of microscopy images](https://mlsanity.com/paper/2506.20649/) (2025)
- [A Systematic Framework for Tabular Data Disentanglement](https://mlsanity.com/paper/2604.07940/) (2026)
- [Transformation Behavior of Images in Latent Space](https://mlsanity.com/paper/2606.24430/) (2026)

## Abstract (arXiv)

With the resurgence of interest in neural networks, representation learning has re-emerged as a central focus in artificial intelligence. Representation learning refers to the discovery of useful encodings of data that make domain-relevant information explicit. Factorial representations identify underlying independent causal factors of variation in data. A factorial representation is compact and faithful, makes the causal factors explicit, and facilitates human interpretation of data. Factorial representations support a variety of applications, including the generation of novel examples, indexing and search, novelty detection, and transfer learning. This article surveys various constraints that encourage a learning algorithm to discover factorial representations. I dichotomize the constraints in terms of unsupervised and supervised inductive bias. Unsupervised inductive biases exploit assumptions about the environment, such as the statistical distribution of factor coefficients, assumptions about the perturbations a factor should be invariant to (e.g. a representation of an object can be invariant to rotation, translation or scaling), and assumptions about how factors are combined to synthesize an observation. Supervised inductive biases are constraints on the representations based on additional information connected to observations. Supervisory labels come in variety of types, which vary in how strongly they constrain the representation, how many factors are labeled, how many observations are labeled, and whether or not we know the associations between the constraints and the factors they are related to. This survey brings together a wide variety of models that all touch on the problem of learning factorial representations and lays out a framework for comparing these models based on the strengths of the underlying supervised and unsupervised inductive biases.