# Estimating Categorical Counterfactuals via Deep Twin Networks

arXiv: https://arxiv.org/abs/2109.01904 · HTML: https://mlsanity.com/paper/2109.01904/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 56 references that are in the map.
- First posted on arXiv 4 Sep 2021.

## Builds on (most cited first, 21 of 21)

- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [Causal Effect Inference with Deep Latent-Variable Models](https://mlsanity.com/paper/1705.08821/) (2017)
- [Learning Representations for Counterfactual Inference](https://mlsanity.com/paper/1605.03661/) (2016)
- [Adapting Neural Networks for the Estimation of Treatment Effects](https://mlsanity.com/paper/1906.02120/) (2019)
- [Deep Structural Causal Models for Tractable Counterfactual Inference](https://mlsanity.com/paper/2006.06485/) (2020)
- [Bayesian Inference of Individualized Treatment Effects using Multi-task Gaussian Processes](https://mlsanity.com/paper/1704.02801/) (2017)
- [CausalGAN: Learning Causal Implicit Generative Models with Adversarial Training](https://mlsanity.com/paper/1709.02023/) (2017)
- [Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Making Systems](https://mlsanity.com/paper/1907.09615/) (2019)
- [Counterfactual Off-Policy Evaluation with Gumbel-Max Structural Causal Models](https://mlsanity.com/paper/1905.05824/) (2019)
- [Probabilities of Causation: Bounds and Identification](https://mlsanity.com/paper/1301.3898/) (2013)
- [Perfect Match: A Simple Method for Learning Representations For Counterfactual Inference With Neural Networks](https://mlsanity.com/paper/1810.00656/) (2018)
- [Explaining Black-Box Algorithms Using Probabilistic Contrastive Counterfactuals](https://mlsanity.com/paper/2103.11972/) (2021)
- [Exploring Counterfactual Explanations Through the Lens of Adversarial Examples: A Theoretical and Empirical Analysis](https://mlsanity.com/paper/2106.09992/) (2021)
- [Counterfactual Representation Learning with Balancing Weights](https://mlsanity.com/paper/2010.12618/) (2020)
- [Monotonic Calibrated Interpolated Look-Up Tables](https://mlsanity.com/paper/1505.06378/) (2015)
- [Counterexample-Guided Learning of Monotonic Neural Networks](https://mlsanity.com/paper/2006.08852/) (2020)
- [Deep Counterfactual Networks with Propensity-Dropout](https://mlsanity.com/paper/1706.05966/) (2017)
- [Learning Functional Causal Models with Generative Neural Networks](https://mlsanity.com/paper/1709.05321/) (2017)
- [NCoRE: Neural Counterfactual Representation Learning for Combinations of Treatments](https://mlsanity.com/paper/2103.11175/) (2021)
- [Causal Inference using Gaussian Processes with Structured Latent Confounders](https://mlsanity.com/paper/2007.07127/) (2020)
- [A nonparametric projection-based estimator for the probability of causation, with application to water sanitation in Kenya](https://mlsanity.com/paper/1810.00767/) (2018)

## Built on by (most cited first, 19 of 19)

- [Causal Machine Learning for Healthcare and Precision Medicine](https://mlsanity.com/paper/2205.11402/) (2022)
- [A Comprehensive Survey of Deep Transfer Learning for Anomaly Detection in Industrial Time Series: Methods, Applications, and Directions](https://mlsanity.com/paper/2307.05638/) (2023)
- [Benchmarking Counterfactual Image Generation](https://mlsanity.com/paper/2403.20287/) (2024)
- [Counterfactual (Non-)identifiability of Learned Structural Causal Models](https://mlsanity.com/paper/2301.09031/) (2023)
- [Partial Counterfactual Identification of Continuous Outcomes with a Curvature Sensitivity Model](https://mlsanity.com/paper/2306.01424/) (2023)
- [Counterfactual Generative Modeling with Variational Causal Inference](https://mlsanity.com/paper/2410.12730/) (2024)
- [Disentangling Causal Effects from Sets of Interventions in the Presence of Unobserved Confounders](https://mlsanity.com/paper/2210.05446/) (2022)
- [Evaluation of Large Language Models via Coupled Token Generation](https://mlsanity.com/paper/2502.01754/) (2025)
- [Contrastive representations of high-dimensional, structured treatments](https://mlsanity.com/paper/2411.19245/) (2024)
- [Causally Steered Diffusion for Automated Video Counterfactual Generation](https://mlsanity.com/paper/2506.14404/) (2025)
- [Counterfactual Identifiability via Dynamic Optimal Transport](https://mlsanity.com/paper/2510.08294/) (2025)
- [Robust Counterfactual Inference in Markov Decision Processes](https://mlsanity.com/paper/2502.13731/) (2025)
- [Local Interference: Removing Interference Bias in Semi-Parametric Causal Models](https://mlsanity.com/paper/2503.18756/) (2025)
- [Incremental Recommendation via Causal Models](https://mlsanity.com/paper/2608.26804/) (2026)
- [Teleporter Theory: A General and Simple Approach for Modeling Cross-World Counterfactual Causality](https://mlsanity.com/paper/2406.11501/) (2024)
- [Unsupervised Structural-Counterfactual Generation under Domain Shift](https://mlsanity.com/paper/2502.12013/) (2025)
- [Quantum oracles give an advantage for identifying classical counterfactuals](https://mlsanity.com/paper/2512.13692/) (2025)
- [RED-Sphere: Hyperspherical Residual Edge Debiasing for Cross-Population Fundus Disease Domain Generalization](https://mlsanity.com/paper/2607.10777/) (2026)
- [Defence against the dark arts of distillation: Unsupervised discovery of subliminal learning](https://mlsanity.com/paper/2610.13536/) (2026)

## Abstract (arXiv)

Counterfactual inference is a powerful tool, capable of solving challenging problems in high-profile sectors. To perform counterfactual inference, one requires knowledge of the underlying causal mechanisms. However, causal mechanisms cannot be uniquely determined from observations and interventions alone. This raises the question of how to choose the causal mechanisms so that resulting counterfactual inference is trustworthy in a given domain. This question has been addressed in causal models with binary variables, but the case of categorical variables remains unanswered. We address this challenge by introducing for causal models with categorical variables the notion of counterfactual ordering, a principle that posits desirable properties causal mechanisms should posses, and prove that it is equivalent to specific functional constraints on the causal mechanisms. To learn causal mechanisms satisfying these constraints, and perform counterfactual inference with them, we introduce deep twin networks. These are deep neural networks that, when trained, are capable of twin network counterfactual inference -- an alternative to the abduction, action, & prediction method. We empirically test our approach on diverse real-world and semi-synthetic data from medicine, epidemiology, and finance, reporting accurate estimation of counterfactual probabilities while demonstrating the issues that arise with counterfactual reasoning when counterfactual ordering is not enforced.