# CausalBench: A Large-scale Benchmark for Network Inference from Single-cell Perturbation Data

arXiv: https://arxiv.org/abs/2210.17283 · HTML: https://mlsanity.com/paper/2210.17283/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 83 references that are in the map.
- First posted on arXiv 31 Oct 2022.

## Builds on (most cited first, 17 of 17)

- [DAGs with NO TEARS: Continuous Optimization for Structure Learning](https://mlsanity.com/paper/1803.01422/) (2018)
- [Differentiable Causal Discovery from Interventional Data](https://mlsanity.com/paper/2007.01754/) (2020)
- [Learning Sparse Nonparametric DAGs](https://mlsanity.com/paper/1909.13189/) (2019)
- [On Wasserstein Two Sample Testing and Related Families of Nonparametric Tests](https://mlsanity.com/paper/1509.02237/) (2015)
- [Beware of the Simulated DAG! Causal Discovery Benchmarks May Be Easy To Game](https://mlsanity.com/paper/2102.13647/) (2021)
- [Learning Neural Causal Models from Unknown Interventions](https://mlsanity.com/paper/1910.01075/) (2019)
- [Learning Counterfactual Representations for Estimating Individual Dose-Response Curves](https://mlsanity.com/paper/1902.00981/) (2019)
- [Amortized Inference for Causal Structure Learning](https://mlsanity.com/paper/2205.12934/) (2022)
- [Permutation-based Causal Inference Algorithms with Interventions](https://mlsanity.com/paper/1705.10220/) (2017)
- [Large-Scale Differentiable Causal Discovery of Factor Graphs](https://mlsanity.com/paper/2206.07824/) (2022)
- [RealCause: Realistic Causal Inference Benchmarking](https://mlsanity.com/paper/2011.15007/) (2020)
- [Learning Neural Causal Models with Active Interventions](https://mlsanity.com/paper/2109.02429/) (2021)
- [Benchmarking Framework for Performance-Evaluation of Causal Inference Analysis](https://mlsanity.com/paper/1802.05046/) (2018)
- [The Case for Evaluating Causal Models Using Interventional Measures and Empirical Data](https://mlsanity.com/paper/1910.05387/) (2019)
- [Validating Causal Inference Methods](https://mlsanity.com/paper/2202.04208/) (2022)
- [GeneDisco: A Benchmark for Experimental Design in Drug Discovery](https://mlsanity.com/paper/2110.11875/) (2021)
- [Size of Interventional Markov Equivalence Classes in Random DAG Models](https://mlsanity.com/paper/1903.02054/) (2019)

## Built on by (most cited first, 29 of 29)

- [CausalTime: Realistically Generated Time-series for Benchmarking of Causal Discovery](https://mlsanity.com/paper/2310.01753/) (2023)
- [CausalRivers -- Scaling up benchmarking of causal discovery for real-world time-series](https://mlsanity.com/paper/2503.17452/) (2025)
- [Sample, estimate, aggregate: A recipe for causal discovery foundation models](https://mlsanity.com/paper/2402.01929/) (2024)
- [The Landscape of Causal Discovery Data: Grounding Causal Discovery in Real-World Applications](https://mlsanity.com/paper/2412.01953/) (2024)
- [Learning DAGs from Data with Few Root Causes](https://mlsanity.com/paper/2305.15936/) (2023)
- [The CausalBench challenge: A machine learning contest for gene network inference from single-cell perturbation data](https://mlsanity.com/paper/2308.15395/) (2023)
- [Gene Regulatory Network Inference in the Presence of Selection Bias and Latent Confounders](https://mlsanity.com/paper/2501.10124/) (2025)
- [Position: Causal Machine Learning Requires Rigorous Synthetic Experiments for Broader Adoption](https://mlsanity.com/paper/2508.08883/) (2025)
- [Causality-oriented robustness: exploiting general noise interventions](https://mlsanity.com/paper/2307.10299/) (2023)
- [Causal Inference in Gene Regulatory Networks with GFlowNet: Towards Scalability in Large Systems](https://mlsanity.com/paper/2310.03579/) (2023)
- [Deriving Causal Order from Single-Variable Interventions: Guarantees & Algorithm](https://mlsanity.com/paper/2405.18314/) (2024)
- [CausalMan: A physics-based simulator for large-scale causality](https://mlsanity.com/paper/2502.12707/) (2025)
- [In-silico biological discovery with large perturbation models](https://mlsanity.com/paper/2503.23535/) (2025)
- [Multi-omics Prediction from High-content Cellular Imaging with Deep Learning](https://mlsanity.com/paper/2306.09391/) (2023)
- [Benchmarking Bayesian Causal Discovery Methods for Downstream Treatment Effect Estimation](https://mlsanity.com/paper/2307.04988/) (2023)
- [Assessing the overall and partial causal well-specification of nonlinear additive noise models](https://mlsanity.com/paper/2310.16502/) (2023)
- [Simulation-based Benchmarking for Causal Structure Learning in Gene Perturbation Experiments](https://mlsanity.com/paper/2407.06015/) (2024)
- [Achievable distributional robustness when the robust risk is only partially identified](https://mlsanity.com/paper/2502.02710/) (2025)
- [IGDA: Interactive Graph Discovery through Large Language Model Agents](https://mlsanity.com/paper/2502.17189/) (2025)
- [Causality-Inspired Robustness for Nonlinear Models via Representation Learning](https://mlsanity.com/paper/2505.12868/) (2025)
- [Efficient Differentiable Discovery of Causal Order](https://mlsanity.com/paper/2410.08787/) (2024)
- [Large Language Models for Zero-shot Inference of Causal Structures in Biology](https://mlsanity.com/paper/2503.04347/) (2025)
- [InfoSEM: A Deep Generative Model with Informative Priors for Gene Regulatory Network Inference](https://mlsanity.com/paper/2503.04483/) (2025)
- [Scalable Temporal Anomaly Causality Discovery in Large Systems: Achieving Computational Efficiency with Binary Anomaly Flag Data](https://mlsanity.com/paper/2412.11800/) (2024)
- [GPO-VAE: Modeling Explainable Gene Perturbation Responses utilizing GRN-Aligned Parameter Optimization](https://mlsanity.com/paper/2501.18973/) (2025)
- [Addressing pitfalls in implicit unobserved confounding synthesis using explicit block hierarchical ancestral sampling](https://mlsanity.com/paper/2503.09194/) (2025)
- [Practical Causal Evaluation Metrics for Biological Networks](https://mlsanity.com/paper/2511.12805/) (2025)
- [When Does Gene Regulatory Network Inference Break? A Controlled Diagnostic Study of Causal and Correlational Methods on Single-Cell Data](https://mlsanity.com/paper/2605.04930/) (2026)
- [Causal ASCEND: Scalable Two-tier Causal Discovery on High Dimensional Multi-omics Data](https://mlsanity.com/paper/2607.04527/) (2026)

## Abstract (arXiv)

Causal inference is a vital aspect of multiple scientific disciplines and is routinely applied to high-impact applications such as medicine. However, evaluating the performance of causal inference methods in real-world environments is challenging due to the need for observations under both interventional and control conditions. Traditional evaluations conducted on synthetic datasets do not reflect the performance in real-world systems. To address this, we introduce CausalBench, a benchmark suite for evaluating network inference methods on real-world interventional data from large-scale single-cell perturbation experiments. CausalBench incorporates biologically-motivated performance metrics, including new distribution-based interventional metrics. A systematic evaluation of state-of-the-art causal inference methods using our CausalBench suite highlights how poor scalability of current methods limits performance. Moreover, methods that use interventional information do not outperform those that only use observational data, contrary to what is observed on synthetic benchmarks. Thus, CausalBench opens new avenues in causal network inference research and provides a principled and reliable way to track progress in leveraging real-world interventional data.