# When Counterfactual Reasoning Fails: Chaos and Real-World Complexity

arXiv: https://arxiv.org/abs/2503.23820 · HTML: https://mlsanity.com/paper/2503.23820/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 36 references that are in the map.
- First posted on arXiv 31 Mar 2025.

## Builds on (most cited first, 18 of 18)

- [CLEVRER: CoLlision Events for Video REpresentation and Reasoning](https://mlsanity.com/paper/1910.01442/) (2019)
- [Counterfactual Off-Policy Evaluation with Gumbel-Max Structural Causal Models](https://mlsanity.com/paper/1905.05824/) (2019)
- [Counterfactual Explanations Can Be Manipulated](https://mlsanity.com/paper/2106.02666/) (2021)
- [The Dangers of Post-hoc Interpretability: Unjustified Counterfactual Explanations](https://mlsanity.com/paper/1907.09294/) (2019)
- [If Only We Had Better Counterfactual Explanations: Five Key Deficits to Rectify in the Evaluation of Counterfactual XAI Techniques](https://mlsanity.com/paper/2103.01035/) (2021)
- [CoPhy: Counterfactual Learning of Physical Dynamics](https://mlsanity.com/paper/1909.12000/) (2019)
- [Time Series Deconfounder: Estimating Treatment Effects over Time in the Presence of Hidden Confounders](https://mlsanity.com/paper/1902.00450/) (2019)
- [Dynamic Visual Reasoning by Learning Differentiable Physics Models from Video and Language](https://mlsanity.com/paper/2110.15358/) (2021)
- [Causal Modeling for Fairness in Dynamical Systems](https://mlsanity.com/paper/1909.09141/) (2019)
- [Counterfactual Explanations in Sequential Decision Making Under Uncertainty](https://mlsanity.com/paper/2107.02776/) (2021)
- [From Deterministic ODEs to Dynamic Structural Causal Models](https://mlsanity.com/paper/1608.08028/) (2016)
- [Causal Modeling of Dynamical Systems](https://mlsanity.com/paper/1803.08784/) (2018)
- [Counterfactual Temporal Point Processes](https://mlsanity.com/paper/2111.07603/) (2021)
- [Filtered-CoPhy: Unsupervised Learning of Counterfactual Physics in Pixel Space](https://mlsanity.com/paper/2202.00368/) (2022)
- [Counterfactual Analysis in Dynamic Latent State Models](https://mlsanity.com/paper/2205.13832/) (2022)
- [A Practical Approach to Causal Inference over Time](https://mlsanity.com/paper/2410.10502/) (2024)
- [Leveraging Structured Biological Knowledge for Counterfactual Inference: a Case Study of Viral Pathogenesis](https://mlsanity.com/paper/2101.05136/) (2021)
- [Counterfactual-based Root Cause Analysis for Dynamical Systems](https://mlsanity.com/paper/2406.08106/) (2024)

## Built on by (most cited first, 1 of 1)

- [Safety, Security, and Cognitive Risks in World Models](https://mlsanity.com/paper/2604.01346/) (2026)

## Abstract (arXiv)

Counterfactual reasoning, a cornerstone of human cognition and decision-making, is often seen as the 'holy grail' of causal learning, with applications ranging from interpreting machine learning models to promoting algorithmic fairness. While counterfactual reasoning has been extensively studied in contexts where the underlying causal model is well-defined, real-world causal modeling is often hindered by model and parameter uncertainty, observational noise, and chaotic behavior. The reliability of counterfactual analysis in such settings remains largely unexplored. In this work, we investigate the limitations of counterfactual reasoning within the framework of Structural Causal Models. Specifically, we empirically investigate counterfactual sequence estimation and highlight cases where it becomes increasingly unreliable. We find that realistic assumptions, such as low degrees of model uncertainty or chaotic dynamics, can result in counterintuitive outcomes, including dramatic deviations between predicted and true counterfactual trajectories. This work urges caution when applying counterfactual reasoning in settings characterized by chaos and uncertainty. Furthermore, it raises the question of whether certain systems may pose fundamental limitations on the ability to answer counterfactual questions about their behavior.