# Entropic Causal Inference

arXiv: https://arxiv.org/abs/1611.04035 · HTML: https://mlsanity.com/paper/1611.04035/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Nov 2016.

## Built on by (most cited first, 27 of 27)

- [On the Identifiability and Estimation of Causal Location-Scale Noise Models](https://mlsanity.com/paper/2210.09054/) (2022)
- [Causal Structure Learning: a Combinatorial Perspective](https://mlsanity.com/paper/2206.01152/) (2022)
- [Perfectly Secure Steganography Using Minimum Entropy Coupling](https://mlsanity.com/paper/2210.14889/) (2022)
- [Multimodal Learning Without Labeled Multimodal Data: Guarantees and Applications](https://mlsanity.com/paper/2306.04539/) (2023)
- [Efficient Bayesian network structure learning via local Markov boundary search](https://mlsanity.com/paper/2110.06082/) (2021)
- [Causal Entropy Optimization](https://mlsanity.com/paper/2208.10981/) (2022)
- [Entropic Causal Inference: Identifiability and Finite Sample Results](https://mlsanity.com/paper/2101.03501/) (2021)
- [Unicorn: Reasoning about Configurable System Performance through the lens of Causality](https://mlsanity.com/paper/2201.08413/) (2022)
- [Obtaining Causal Information by Merging Datasets with MAXENT](https://mlsanity.com/paper/2107.07640/) (2021)
- [Causal versions of Maximum Entropy and Principle of Insufficient Reason](https://mlsanity.com/paper/2102.03906/) (2021)
- [Towards Semantic Communication Protocols for 6G: From Protocol Learning to Language-Oriented Approaches](https://mlsanity.com/paper/2310.09506/) (2023)
- [Fundamental Limits of Perfect Concept Erasure](https://mlsanity.com/paper/2503.20098/) (2025)
- [Minimum Entropy Coupling with Bottleneck](https://mlsanity.com/paper/2410.21666/) (2024)
- [On Causality in Domain Adaptation and Semi-Supervised Learning: an Information-Theoretic Analysis for Parametric Models](https://mlsanity.com/paper/2205.04641/) (2022)
- [Do learned representations respect causal relationships?](https://mlsanity.com/paper/2204.00762/) (2022)
- [Approximate Causal Effect Identification under Weak Confounding](https://mlsanity.com/paper/2306.13242/) (2023)
- [Cross-validating causal discovery via Leave-One-Variable-Out](https://mlsanity.com/paper/2411.05625/) (2024)
- [Entropic Causal Inference: Graph Identifiability](https://mlsanity.com/paper/2509.16463/) (2025)
- [Learning to Match Unpaired Data with Minimum Entropy Coupling](https://mlsanity.com/paper/2503.08501/) (2025)
- [Provable Secure Steganography Based on Adaptive Dynamic Sampling](https://mlsanity.com/paper/2504.12579/) (2025)
- [GLACIAL: Granger and Learning-based Causality Analysis for Longitudinal Imaging Studies](https://mlsanity.com/paper/2210.07416/) (2022)
- [Distinguishing Cause from Effect on Categorical Data: The Uniform Channel Model](https://mlsanity.com/paper/2303.08572/) (2023)
- [Ordering-based Causal Discovery via Generalized Score Matching](https://mlsanity.com/paper/2601.16249/) (2026)
- [Variation-based Cause Effect Identification](https://mlsanity.com/paper/2211.12016/) (2022)
- [Detection of Unobserved Common Causes based on NML Code in Discrete, Mixed, and Continuous Variables](https://mlsanity.com/paper/2403.06499/) (2024)
- [Cross-Domain Lossy Compression via Constrained Minimum Entropy Coupling](https://mlsanity.com/paper/2605.09833/) (2026)
- [Learning from Totally Unpaired Data via Minimum Conditional Entropy Coupling](https://mlsanity.com/paper/2605.16229/) (2026)

## Abstract (arXiv)

We consider the problem of identifying the causal direction between two discrete random variables using observational data. Unlike previous work, we keep the most general functional model but make an assumption on the unobserved exogenous variable: Inspired by Occam's razor, we assume that the exogenous variable is simple in the true causal direction. We quantify simplicity using R'enyi entropy. Our main result is that, under natural assumptions, if the exogenous variable has low H0 entropy (cardinality) in the true direction, it must have high H0 entropy in the wrong direction. We establish several algorithmic hardness results about estimating the minimum entropy exogenous variable. We show that the problem of finding the exogenous variable with minimum entropy is equivalent to the problem of finding minimum joint entropy given n marginal distributions, also known as minimum entropy coupling problem. We propose an efficient greedy algorithm for the minimum entropy coupling problem, that for n=2 provably finds a local optimum. This gives a greedy algorithm for finding the exogenous variable with minimum H1 (Shannon Entropy). Our greedy entropy-based causal inference algorithm has similar performance to the state of the art additive noise models in real datasets. One advantage of our approach is that we make no use of the values of random variables but only their distributions. Our method can therefore be used for causal inference for both ordinal and also categorical data, unlike additive noise models.