# ICE-BeeM: Identifiable Conditional Energy-Based Deep Models Based on Nonlinear ICA

arXiv: https://arxiv.org/abs/2002.11537 · HTML: https://mlsanity.com/paper/2002.11537/

## Key facts

- Cited by 102 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Feb 2020.

## Built on by (most cited first, 30 of 102)

- [Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style](https://mlsanity.com/paper/2106.04619/) (2021)
- [Contrastive Learning Inverts the Data Generating Process](https://mlsanity.com/paper/2102.08850/) (2021)
- [Interventional Causal Representation Learning](https://mlsanity.com/paper/2209.11924/) (2022)
- [CITRIS: Causal Identifiability from Temporal Intervened Sequences](https://mlsanity.com/paper/2202.03169/) (2022)
- [Nonparametric Identifiability of Causal Representations from Unknown Interventions](https://mlsanity.com/paper/2306.00542/) (2023)
- [Learning Linear Causal Representations from Interventions under General Nonlinear Mixing](https://mlsanity.com/paper/2306.02235/) (2023)
- [Multi-View Causal Representation Learning with Partial Observability](https://mlsanity.com/paper/2311.04056/) (2023)
- [Unsupervised Learning of Compositional Energy Concepts](https://mlsanity.com/paper/2111.03042/) (2021)
- [Identifiability of deep generative models without auxiliary information](https://mlsanity.com/paper/2206.10044/) (2022)
- [Causal Representation Learning from Multiple Distributions: A General Setting](https://mlsanity.com/paper/2402.05052/) (2024)
- [Weakly Supervised Representation Learning with Sparse Perturbations](https://mlsanity.com/paper/2206.01101/) (2022)
- [Disentangling Identifiable Features from Noisy Data with Structured Nonlinear ICA](https://mlsanity.com/paper/2106.09620/) (2021)
- [Visual Representation Learning Does Not Generalize Strongly Within the Same Domain](https://mlsanity.com/paper/2107.08221/) (2021)
- [Identifiability of latent-variable and structural-equation models: from linear to nonlinear](https://mlsanity.com/paper/2302.02672/) (2023)
- [Additive Decoders for Latent Variables Identification and Cartesian-Product Extrapolation](https://mlsanity.com/paper/2307.02598/) (2023)
- [Posterior Collapse and Latent Variable Non-identifiability](https://mlsanity.com/paper/2301.00537/) (2023)
- [Causal Component Analysis](https://mlsanity.com/paper/2305.17225/) (2023)
- [Score-based Causal Representation Learning with Interventions](https://mlsanity.com/paper/2301.08230/) (2023)
- [Nonlinear Independent Component Analysis for Principled Disentanglement in Unsupervised Deep Learning](https://mlsanity.com/paper/2303.16535/) (2023)
- [Provably Learning Object-Centric Representations](https://mlsanity.com/paper/2305.14229/) (2023)
- [Synergies between Disentanglement and Sparsity: Generalization and Identifiability in Multi-Task Learning](https://mlsanity.com/paper/2211.14666/) (2022)
- [Nonlinear Invariant Risk Minimization: A Causal Approach](https://mlsanity.com/paper/2102.12353/) (2021)
- [Provable Compositional Generalization for Object-Centric Learning](https://mlsanity.com/paper/2310.05327/) (2023)
- [BISCUIT: Causal Representation Learning from Binary Interactions](https://mlsanity.com/paper/2306.09643/) (2023)
- [Subspace Identification for Multi-Source Domain Adaptation](https://mlsanity.com/paper/2310.04723/) (2023)
- [Learning nonparametric latent causal graphs with unknown interventions](https://mlsanity.com/paper/2306.02899/) (2023)
- [General Identifiability and Achievability for Causal Representation Learning](https://mlsanity.com/paper/2310.15450/) (2023)
- [Generalizing Nonlinear ICA Beyond Structural Sparsity](https://mlsanity.com/paper/2311.00866/) (2023)
- [Causal Representation Learning Made Identifiable by Grouping of Observational Variables](https://mlsanity.com/paper/2310.15709/) (2023)
- [Unifying Causal Representation Learning with the Invariance Principle](https://mlsanity.com/paper/2409.02772/) (2024)

## Abstract (arXiv)

We consider the identifiability theory of probabilistic models and establish sufficient conditions under which the representations learned by a very broad family of conditional energy-based models are unique in function space, up to a simple transformation. In our model family, the energy function is the dot-product between two feature extractors, one for the dependent variable, and one for the conditioning variable. We show that under mild conditions, the features are unique up to scaling and permutation. Our results extend recent developments in nonlinear ICA, and in fact, they lead to an important generalization of ICA models. In particular, we show that our model can be used for the estimation of the components in the framework of Independently Modulated Component Analysis (IMCA), a new generalization of nonlinear ICA that relaxes the independence assumption. A thorough empirical study shows that representations learned by our model from real-world image datasets are identifiable, and improve performance in transfer learning and semi-supervised learning tasks.