# Self-Distilled Disentangled Learning for Counterfactual Prediction

arXiv: https://arxiv.org/abs/2406.05855 · HTML: https://mlsanity.com/paper/2406.05855/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 50 references that are in the map.
- First posted on arXiv 9 Jun 2024.

## Builds on (most cited first, 15 of 15)

- [Disentangling by Factorising](https://mlsanity.com/paper/1802.05983/) (2018)
- [On Variational Bounds of Mutual Information](https://mlsanity.com/paper/1905.06922/) (2019)
- [CLUB: A Contrastive Log-ratio Upper Bound of Mutual Information](https://mlsanity.com/paper/2006.12013/) (2020)
- [Bayesian Inference of Individualized Treatment Effects using Multi-task Gaussian Processes](https://mlsanity.com/paper/1704.02801/) (2017)
- [Domain Adaptation by Using Causal Inference to Predict Invariant Conditional Distributions](https://mlsanity.com/paper/1707.06422/) (2017)
- [Kernel Instrumental Variable Regression](https://mlsanity.com/paper/1906.00232/) (2019)
- [The Blessings of Multiple Causes](https://mlsanity.com/paper/1805.06826/) (2018)
- [Learning Deep Features in Instrumental Variable Regression](https://mlsanity.com/paper/2010.07154/) (2020)
- [Treatment effect estimation with disentangled latent factors](https://mlsanity.com/paper/2001.10652/) (2020)
- [Using Embeddings to Correct for Unobserved Confounding in Networks](https://mlsanity.com/paper/1902.04114/) (2019)
- [Auto IV: Counterfactual Prediction via Automatic Instrumental Variable Decomposition](https://mlsanity.com/paper/2107.05884/) (2021)
- [Learning Disentangled Representations for Counterfactual Regression via Mutual Information Minimization](https://mlsanity.com/paper/2206.01022/) (2022)
- [Learning Decomposed Representation for Counterfactual Inference](https://mlsanity.com/paper/2006.07040/) (2020)
- [Cycle-Balanced Representation Learning For Counterfactual Inference](https://mlsanity.com/paper/2110.15484/) (2021)
- [Effects of Multi-Aspect Online Reviews with Unobserved Confounders: Estimation and Implication](https://mlsanity.com/paper/2110.01746/) (2021)

## Built on by (most cited first, 4 of 4)

- [DrunkAgent: Stealthy Memory Corruption in LLM-Powered Recommender Agents](https://mlsanity.com/paper/2503.23804/) (2025)
- [Causality-aligned Prompt Learning via Diffusion-based Counterfactual Generation](https://mlsanity.com/paper/2507.19882/) (2025)
- [Uncertainty-Guided LLM Semantic Augmentation for Heterogeneous Treatment Effect Estimation](https://mlsanity.com/paper/2607.26599/) (2026)
- [Decouple to Forget: Semantic Mediation for Sensitive Unlearning in Large Language Models](https://mlsanity.com/paper/2610.17654/) (2026)

## Abstract (arXiv)

The advancements in disentangled representation learning significantly enhance the accuracy of counterfactual predictions by granting precise control over instrumental variables, confounders, and adjustable variables. An appealing method for achieving the independent separation of these factors is mutual information minimization, a task that presents challenges in numerous machine learning scenarios, especially within high-dimensional spaces. To circumvent this challenge, we propose the Self-Distilled Disentanglement framework, referred to as SD2. Grounded in information theory, it ensures theoretically sound independent disentangled representations without intricate mutual information estimator designs for high-dimensional representations. Our comprehensive experiments, conducted on both synthetic and real-world datasets, confirms the effectiveness of our approach in facilitating counterfactual inference in the presence of both observed and unobserved confounders.