# Improving Visualization Interpretation Using Counterfactuals

arXiv: https://arxiv.org/abs/2107.10309 · HTML: https://mlsanity.com/paper/2107.10309/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jul 2021.

## Built on by (most cited first, 10 of 10)

- [Visual Analytics For Machine Learning: A Data Perspective Survey](https://mlsanity.com/paper/2307.07712/) (2023)
- [Understanding the Effect of Counterfactual Explanations on Trust and Reliance on AI for Human-AI Collaborative Clinical Decision Making](https://mlsanity.com/paper/2308.04375/) (2023)
- [Visual Analytics for Efficient Image Exploration and User-Guided Image Captioning](https://mlsanity.com/paper/2311.01016/) (2023)
- [Visual Analysis of Multi-outcome Causal Graphs](https://mlsanity.com/paper/2408.02679/) (2024)
- [ESCAPE: Countering Systematic Errors from Machine's Blind Spots via Interactive Visual Analysis](https://mlsanity.com/paper/2303.09657/) (2023)
- [Leveraging MIMIC Datasets for Better Digital Health: A Review on Open Problems, Progress Highlights, and Future Promises](https://mlsanity.com/paper/2506.12808/) (2025)
- [EXPRTS: Exploring and Probing the Robustness of Time Series Forecasting Models](https://mlsanity.com/paper/2403.03508/) (2024)
- [XplainAct: Visualization for Personalized Intervention Insights](https://mlsanity.com/paper/2507.14767/) (2025)
- [Human-Guided Causal Knowledge Injection for Virtual Cells](https://mlsanity.com/paper/2608.08430/) (2026)
- [SocialFiVis: A Visual Analytics Sandbox for LLM-Grounded Multi-Agent Simulation in Social Finance](https://mlsanity.com/paper/2608.08497/) (2026)

## Abstract (arXiv)

Complex, high-dimensional data is used in a wide range of domains to explore problems and make decisions. Analysis of high-dimensional data, however, is vulnerable to the hidden influence of confounding variables, especially as users apply ad hoc filtering operations to visualize only specific subsets of an entire dataset. Thus, visual data-driven analysis can mislead users and encourage mistaken assumptions about causality or the strength of relationships between features. This work introduces a novel visual approach designed to reveal the presence of confounding variables via counterfactual possibilities during visual data analysis. It is implemented in CoFact, an interactive visualization prototype that determines and visualizes textitcounterfactual subsets to better support user exploration of feature relationships. Using publicly available datasets, we conducted a controlled user study to demonstrate the effectiveness of our approach; the results indicate that users exposed to counterfactual visualizations formed more careful judgments about feature-to-outcome relationships.