# Sense and Sensitivity Analysis: Simple Post-Hoc Analysis of Bias Due to Unobserved Confounding

arXiv: https://arxiv.org/abs/2003.01747 · HTML: https://mlsanity.com/paper/2003.01747/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Mar 2020.

## Built on by (most cited first, 25 of 25)

- [Quantifying Ignorance in Individual-Level Causal-Effect Estimates under Hidden Confounding](https://mlsanity.com/paper/2103.04850/) (2021)
- [Long Story Short: Omitted Variable Bias in Causal Machine Learning](https://mlsanity.com/paper/2112.13398/) (2021)
- [DoWhy: Addressing Challenges in Expressing and Validating Causal Assumptions](https://mlsanity.com/paper/2108.13518/) (2021)
- [Prescriptive Process Monitoring for Cost-Aware Cycle Time Reduction](https://mlsanity.com/paper/2105.07111/) (2021)
- [A Diachronic Analysis of Paradigm Shifts in NLP Research: When, How, and Why?](https://mlsanity.com/paper/2305.12920/) (2023)
- [Partial Identification with Noisy Covariates: A Robust Optimization Approach](https://mlsanity.com/paper/2202.10665/) (2022)
- [Hidden yet quantifiable: A lower bound for confounding strength using randomized trials](https://mlsanity.com/paper/2312.03871/) (2023)
- [Partial Identification of Dose Responses with Hidden Confounders](https://mlsanity.com/paper/2204.11206/) (2022)
- [Quantitative probing: Validating causal models using quantitative domain knowledge](https://mlsanity.com/paper/2209.03013/) (2022)
- [Non-parametric identifiability and sensitivity analysis of synthetic control models](https://mlsanity.com/paper/2301.07656/) (2023)
- [Front-door Adjustment Beyond Markov Equivalence with Limited Graph Knowledge](https://mlsanity.com/paper/2306.11008/) (2023)
- [Confounding-Robust Policy Improvement with Human-AI Teams](https://mlsanity.com/paper/2310.08824/) (2023)
- [Spillover Detection for Donor Selection in Synthetic Control Models](https://mlsanity.com/paper/2406.11399/) (2024)
- [Causal Inference from Small High-dimensional Datasets](https://mlsanity.com/paper/2205.09281/) (2022)
- [PWSHAP: A Path-Wise Explanation Model for Targeted Variables](https://mlsanity.com/paper/2306.14672/) (2023)
- [Some Reflections on Drawing Causal Inference using Textual Data: Parallels Between Human Subjects and Organized Texts](https://mlsanity.com/paper/2202.00848/) (2022)
- [ρ-GNF: A Copula-based Sensitivity Analysis to Unobserved Confounding Using Normalizing Flows](https://mlsanity.com/paper/2209.07111/) (2022)
- [Sensitivity Analysis to Unobserved Confounding with Copula-based Normalizing Flows](https://mlsanity.com/paper/2508.08752/) (2025)
- [Technology Readiness Levels for Machine Learning Systems](https://mlsanity.com/paper/2101.03989/) (2021)
- [Externally Valid Policy Evaluation Combining Trial and Observational Data](https://mlsanity.com/paper/2310.14763/) (2023)
- [A/B testing under Interference with Partial Network Information](https://mlsanity.com/paper/2404.10547/) (2024)
- [Conditional Average Treatment Effect Estimation Under Hidden Confounders](https://mlsanity.com/paper/2506.12304/) (2025)
- [Mitigating Hidden Confounding by Progressive Confounder Imputation via Large Language Models](https://mlsanity.com/paper/2507.02928/) (2025)
- [Bayesian Sensitivity of Causal Inference Estimators under Evidence-Based Priors](https://mlsanity.com/paper/2605.07993/) (2026)
- [Causal Machine Learning Is Not a Panacea: A Roadmap for Observational Causal Inference in Health](https://mlsanity.com/paper/2605.20782/) (2026)

## Abstract (arXiv)

It is a truth universally acknowledged that an observed association without known mechanism must be in want of a causal estimate. However, causal estimation from observational data often relies on the (untestable) assumption of `no unobserved confounding'. Violations of this assumption can induce bias in effect estimates. In principle, such bias could invalidate or reverse the conclusions of a study. However, in some cases, we might hope that the influence of unobserved confounders is weak relative to a `large' estimated effect, so the qualitative conclusions are robust to bias from unobserved confounding. The purpose of this paper is to develop Austen plots, a sensitivity analysis tool to aid such judgments by making it easier to reason about potential bias induced by unobserved confounding. We formalize confounding strength in terms of how strongly the confounder influences treatment assignment and outcome. For a target level of bias, an Austen plot shows the minimum values of treatment and outcome influence required to induce that level of bias. Domain experts can then make subjective judgments about whether such strong confounders are plausible. To aid this judgment, the Austen plot additionally displays the estimated influence strength of (groups of) the observed covariates. Austen plots generalize the classic sensitivity analysis approach of Imbens [Imb03]. Critically, Austen plots allow any approach for modeling the observed data and producing the initial estimate. We illustrate the tool by assessing biases for several real causal inference problems, using a variety of machine learning approaches for the initial data analysis. Code is available at https://github.com/anishazaveri/austenplots