# Learning Deep Disentangled Embeddings with the F-Statistic Loss

arXiv: https://arxiv.org/abs/1802.05312 · HTML: https://mlsanity.com/paper/1802.05312/

## Key facts

- Cited by 101 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Feb 2018.

## Built on by (most cited first, 30 of 101)

- [Towards Causal Representation Learning](https://mlsanity.com/paper/2102.11107/) (2021)
- [Generative Adversarial Transformers](https://mlsanity.com/paper/2103.01209/) (2021)
- [Disentangled Representation Learning](https://mlsanity.com/paper/2211.11695/) (2022)
- [Generalization and Robustness Implications in Object-Centric Learning](https://mlsanity.com/paper/2107.00637/) (2021)
- [Deep Gait Recognition: A Survey](https://mlsanity.com/paper/2102.09546/) (2021)
- [Function Classes for Identifiable Nonlinear Independent Component Analysis](https://mlsanity.com/paper/2208.06406/) (2022)
- [Vector-Decomposed Disentanglement for Domain-Invariant Object Detection](https://mlsanity.com/paper/2108.06685/) (2021)
- [Self-Supervised Learning Disentangled Group Representation as Feature](https://mlsanity.com/paper/2110.15255/) (2021)
- [Investigating Explainability of Generative AI for Code through Scenario-based Design](https://mlsanity.com/paper/2202.04903/) (2022)
- [Learning Disentangled Representations in the Imaging Domain](https://mlsanity.com/paper/2108.12043/) (2021)
- [A Framework for Learning Ante-hoc Explainable Models via Concepts](https://mlsanity.com/paper/2108.11761/) (2021)
- [von Mises-Fisher Loss: An Exploration of Embedding Geometries for Supervised Learning](https://mlsanity.com/paper/2103.15718/) (2021)
- [Disentanglement via Latent Quantization](https://mlsanity.com/paper/2305.18378/) (2023)
- [Disentanglement of Correlated Factors via Hausdorff Factorized Support](https://mlsanity.com/paper/2210.07347/) (2022)
- [Neural Systematic Binder](https://mlsanity.com/paper/2211.01177/) (2022)
- [MoStGAN-V: Video Generation with Temporal Motion Styles](https://mlsanity.com/paper/2304.02777/) (2023)
- [Text-Video Retrieval with Disentangled Conceptualization and Set-to-Set Alignment](https://mlsanity.com/paper/2305.12218/) (2023)
- [Disentanglement with Biological Constraints: A Theory of Functional Cell Types](https://mlsanity.com/paper/2210.01768/) (2022)
- [Self-Adversarial Disentangling for Specific Domain Adaptation](https://mlsanity.com/paper/2108.03553/) (2021)
- [Towards Robust Metrics for Concept Representation Evaluation](https://mlsanity.com/paper/2301.10367/) (2023)
- [Symmetry-Based Representations for Artificial and Biological General Intelligence](https://mlsanity.com/paper/2203.09250/) (2022)
- [DCI-ES: An Extended Disentanglement Framework with Connections to Identifiability](https://mlsanity.com/paper/2210.00364/) (2022)
- [On Causally Disentangled Representations](https://mlsanity.com/paper/2112.05746/) (2021)
- [Algorithm Fairness in AI for Medicine and Healthcare](https://mlsanity.com/paper/2110.00603/) (2021)
- [Local Disentanglement in Variational Auto-Encoders Using Jacobian L1 Regularization](https://mlsanity.com/paper/2106.02923/) (2021)
- [Attri-VAE: attribute-based interpretable representations of medical images with variational autoencoders](https://mlsanity.com/paper/2203.10417/) (2022)
- [Interpretable Molecular Graph Generation via Monotonic Constraints](https://mlsanity.com/paper/2203.00412/) (2022)
- [Disentangled Representations from Non-Disentangled Models](https://mlsanity.com/paper/2102.06204/) (2021)
- [Towards a Unified Framework of Contrastive Learning for Disentangled Representations](https://mlsanity.com/paper/2311.04774/) (2023)
- [Deciphering the Role of Representation Disentanglement: Investigating Compositional Generalization in CLIP Models](https://mlsanity.com/paper/2407.05897/) (2024)

## Abstract (arXiv)

Deep-embedding methods aim to discover representations of a domain that make explicit the domain's class structure and thereby support few-shot learning. Disentangling methods aim to make explicit compositional or factorial structure. We combine these two active but independent lines of research and propose a new paradigm suitable for both goals. We propose and evaluate a novel loss function based on the F statistic, which describes the separation of two or more distributions. By ensuring that distinct classes are well separated on a subset of embedding dimensions, we obtain embeddings that are useful for few-shot learning. By not requiring separation on all dimensions, we encourage the discovery of disentangled representations. Our embedding method matches or beats state-of-the-art, as evaluated by performance on recall@k and few-shot learning tasks. Our method also obtains performance superior to a variety of alternatives on disentangling, as evaluated by two key properties of a disentangled representation: modularity and explicitness. The goal of our work is to obtain more interpretable, manipulable, and generalizable deep representations of concepts and categories.