# Contrastive Mixture of Posteriors for Counterfactual Inference, Data Integration and Fairness

arXiv: https://arxiv.org/abs/2106.08161 · HTML: https://mlsanity.com/paper/2106.08161/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 79 references that are in the map.
- First posted on arXiv 15 Jun 2021.

## Builds on (most cited first, 21 of 21)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Quantifying the Carbon Emissions of Machine Learning](https://mlsanity.com/paper/1910.09700/) (2019)
- [Variational Autoencoders and Nonlinear ICA: A Unifying Framework](https://mlsanity.com/paper/1907.04809/) (2019)
- [Causal Effect Inference with Deep Latent-Variable Models](https://mlsanity.com/paper/1705.08821/) (2017)
- [Learning Representations for Counterfactual Inference](https://mlsanity.com/paper/1605.03661/) (2016)
- [VAE with a VampPrior](https://mlsanity.com/paper/1705.07120/) (2017)
- [The Variational Fair Autoencoder](https://mlsanity.com/paper/1511.00830/) (2015)
- [Avoiding Discrimination through Causal Reasoning](https://mlsanity.com/paper/1706.02744/) (2017)
- [Path-Specific Counterfactual Fairness](https://mlsanity.com/paper/1802.08139/) (2018)
- [Variational Autoencoders Pursue PCA Directions (by Accident)](https://mlsanity.com/paper/1812.06775/) (2018)
- [A causal framework for discovering and removing direct and indirect discrimination](https://mlsanity.com/paper/1611.07509/) (2016)
- [A Unified Stochastic Gradient Approach to Designing Bayesian-Optimal Experiments](https://mlsanity.com/paper/1911.00294/) (2019)
- [Implicit Maximum Likelihood Estimation](https://mlsanity.com/paper/1809.09087/) (2018)
- [Counterfactual Fairness with Disentangled Causal Effect Variational Autoencoder](https://mlsanity.com/paper/2011.11878/) (2020)
- [Information Constraints on Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1805.08672/) (2018)
- [Counterfactual Reasoning for Fair Clinical Risk Prediction](https://mlsanity.com/paper/1907.06260/) (2019)
- [Neural Decomposition: Functional ANOVA with Variational Autoencoders](https://mlsanity.com/paper/2006.14293/) (2020)
- [Conditional out-of-sample generation for unpaired data using trVAE](https://mlsanity.com/paper/1910.01791/) (2019)
- [Adjusting for Confounding in Unsupervised Latent Representations of Images](https://mlsanity.com/paper/1811.06498/) (2018)

## Built on by (most cited first, 3 of 3)

- [Toward Operationalizing Pipeline-aware ML Fairness: A Research Agenda for Developing Practical Guidelines and Tools](https://mlsanity.com/paper/2309.17337/) (2023)
- [Propensity Score Alignment of Unpaired Multimodal Data](https://mlsanity.com/paper/2404.01595/) (2024)
- [Curiosity in Hindsight: Intrinsic Exploration in Stochastic Environments](https://mlsanity.com/paper/2211.10515/) (2022)

## Abstract (arXiv)

Learning meaningful representations of data that can address challenges such as batch effect correction and counterfactual inference is a central problem in many domains including computational biology. Adopting a Conditional VAE framework, we show that marginal independence between the representation and a condition variable plays a key role in both of these challenges. We propose the Contrastive Mixture of Posteriors (CoMP) method that uses a novel misalignment penalty defined in terms of mixtures of the variational posteriors to enforce this independence in latent space. We show that CoMP has attractive theoretical properties compared to previous approaches, and we prove counterfactual identifiability of CoMP under additional assumptions. We demonstrate state-of-the-art performance on a set of challenging tasks including aligning human tumour samples with cancer cell-lines, predicting transcriptome-level perturbation responses, and batch correction on single-cell RNA sequencing data. We also find parallels to fair representation learning and demonstrate that CoMP is competitive on a common task in the field.