# Counterfactual Augmentation for Multimodal Learning Under Presentation Bias

arXiv: https://arxiv.org/abs/2305.14083 · HTML: https://mlsanity.com/paper/2305.14083/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 32 references that are in the map.
- First posted on arXiv 23 May 2023.

## Builds on (most cited first, 19 of 19)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [Semantics derived automatically from language corpora contain human-like biases](https://mlsanity.com/paper/1608.07187/) (2016)
- [Causal Effect Inference with Deep Latent-Variable Models](https://mlsanity.com/paper/1705.08821/) (2017)
- [Adapting Neural Networks for the Estimation of Treatment Effects](https://mlsanity.com/paper/1906.02120/) (2019)
- [Recommendations as Treatments: Debiasing Learning and Evaluation](https://mlsanity.com/paper/1602.05352/) (2016)
- [Unbiased Learning-to-Rank with Biased Feedback](https://mlsanity.com/paper/1608.04468/) (2016)
- [Causal Embeddings for Recommendation](https://mlsanity.com/paper/1706.07639/) (2017)
- [Robustness to Spurious Correlations in Text Classification via Automatically Generated Counterfactuals](https://mlsanity.com/paper/2012.10040/) (2020)
- [Unbiased Learning to Rank with Unbiased Propensity Estimation](https://mlsanity.com/paper/1804.05938/) (2018)
- [What are the biases in my word embedding?](https://mlsanity.com/paper/1812.08769/) (2018)
- [Exploring the Efficacy of Automatically Generated Counterfactuals for Sentiment Analysis](https://mlsanity.com/paper/2106.15231/) (2021)
- [Benchmarking Multimodal AutoML for Tabular Data with Text Fields](https://mlsanity.com/paper/2111.02705/) (2021)
- [Do Offline Metrics Predict Online Performance in Recommender Systems?](https://mlsanity.com/paper/2011.07931/) (2020)
- [NeuroCounterfactuals: Beyond Minimal-Edit Counterfactuals for Richer Data Augmentation](https://mlsanity.com/paper/2210.12365/) (2022)
- [Human Interaction with Recommendation Systems](https://mlsanity.com/paper/1703.00535/) (2017)
- [Breaking Feedback Loops in Recommender Systems with Causal Inference](https://mlsanity.com/paper/2207.01616/) (2022)
- [Correcting the User Feedback-Loop Bias for Recommendation Systems](https://mlsanity.com/paper/2109.06037/) (2021)

## Built on by (most cited first, 2 of 2)

- [When Words Speak Louder than Images: Towards Understanding Language Bias in Vision-Language Models](https://mlsanity.com/paper/2609.35272/) (2026)
- [DUALVIEW: Leveraging Broad and Output-Specific Evidence for LLM-based Multi-Output Text Regression](https://mlsanity.com/paper/2610.19206/) (2026)

## Abstract (arXiv)

In real-world machine learning systems, labels are often derived from user behaviors that the system wishes to encourage. Over time, new models must be trained as new training examples and features become available. However, feedback loops between users and models can bias future user behavior, inducing a presentation bias in the labels that compromises the ability to train new models. In this paper, we propose counterfactual augmentation, a novel causal method for correcting presentation bias using generated counterfactual labels. Our empirical evaluations demonstrate that counterfactual augmentation yields better downstream performance compared to both uncorrected models and existing bias-correction methods. Model analyses further indicate that the generated counterfactuals align closely with true counterfactuals in an oracle setting.