# Large-scale empirical validation of Bayesian Network structure learning algorithms with noisy data

arXiv: https://arxiv.org/abs/2005.09020 · HTML: https://mlsanity.com/paper/2005.09020/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 May 2020.

## Built on by (most cited first, 29 of 29)

- [A survey of Bayesian Network structure learning](https://mlsanity.com/paper/2109.11415/) (2021)
- [Causal Discovery from Temporal Data: An Overview and New Perspectives](https://mlsanity.com/paper/2303.10112/) (2023)
- [The impact of prior knowledge on causal structure learning](https://mlsanity.com/paper/2102.00473/) (2021)
- [Marrying Fairness and Explainability in Supervised Learning](https://mlsanity.com/paper/2204.02947/) (2022)
- [textttcausalAssembly: Generating Realistic Production Data for Benchmarking Causal Discovery](https://mlsanity.com/paper/2306.10816/) (2023)
- [Greedy structure learning from data that contain systematic missing values](https://mlsanity.com/paper/2107.04184/) (2021)
- [Benchpress: A Scalable and Versatile Workflow for Benchmarking Structure Learning Algorithms](https://mlsanity.com/paper/2107.03863/) (2021)
- [bnRep: A repository of Bayesian networks from the academic literature](https://mlsanity.com/paper/2409.19158/) (2024)
- [Effective and efficient structure learning with pruning and model averaging strategies](https://mlsanity.com/paper/2112.00398/) (2021)
- [Open problems in causal structure learning: A case study of COVID-19 in the UK](https://mlsanity.com/paper/2305.03859/) (2023)
- [The Impact of Variable Ordering on Bayesian Network Structure Learning](https://mlsanity.com/paper/2206.08952/) (2022)
- [Causal discovery using dynamically requested knowledge](https://mlsanity.com/paper/2310.11154/) (2023)
- [Efficient SAGE Estimation via Causal Structure Learning](https://mlsanity.com/paper/2304.03113/) (2023)
- [Investigating potential causes of Sepsis with Bayesian network structure learning](https://mlsanity.com/paper/2406.09207/) (2024)
- [The Dual PC Algorithm and the Role of Gaussianity for Structure Learning of Bayesian Networks](https://mlsanity.com/paper/2112.09036/) (2021)
- [Discovery and density estimation of latent confounders in Bayesian networks with evidence lower bound](https://mlsanity.com/paper/2206.05490/) (2022)
- [Causal DAG Summarization (Full Version)](https://mlsanity.com/paper/2504.14937/) (2025)
- [Dynamic Expert-Guided Model Averaging for Causal Discovery](https://mlsanity.com/paper/2601.16715/) (2026)
- [Hybrid Bayesian network discovery with latent variables by scoring multiple interventions](https://mlsanity.com/paper/2112.10574/) (2021)
- [Tuning structure learning algorithms with out-of-sample and resampling strategies](https://mlsanity.com/paper/2306.13932/) (2023)
- [Machine learning in the social and health sciences](https://mlsanity.com/paper/2106.10716/) (2021)
- [High-Dimensional Inference in Bayesian Networks](https://mlsanity.com/paper/2112.09217/) (2021)
- [Parallel Sampling for Efficient High-dimensional Bayesian Network Structure Learning](https://mlsanity.com/paper/2202.09691/) (2022)
- [A Comprehensively Improved Hybrid Algorithm for Learning Bayesian Networks: Multiple Compound Memory Erasing](https://mlsanity.com/paper/2212.03103/) (2022)
- [Using machine learning to understand causal relationships between urban form and travel CO2 emissions across continents](https://mlsanity.com/paper/2308.16599/) (2023)
- [Distributionally Robust Skeleton Learning of Discrete Bayesian Networks](https://mlsanity.com/paper/2311.06117/) (2023)
- [Data-Driven Bayesian Network Models of Hurricane Evacuation Decision Making](https://mlsanity.com/paper/2311.10228/) (2023)
- [Modeling Psychological Profiles in Volleyball via Mixed-Type Bayesian Networks](https://mlsanity.com/paper/2509.22111/) (2025)
- [Time series causal discovery with variable lags](https://mlsanity.com/paper/2605.04081/) (2026)

## Abstract (arXiv)

Numerous Bayesian Network (BN) structure learning algorithms have been proposed in the literature over the past few decades. Each publication makes an empirical or theoretical case for the algorithm proposed in that publication and results across studies are often inconsistent in their claims about which algorithm is 'best'. This is partly because there is no agreed evaluation approach to determine their effectiveness. Moreover, each algorithm is based on a set of assumptions, such as complete data and causal sufficiency, and tend to be evaluated with data that conforms to these assumptions, however unrealistic these assumptions may be in the real world. As a result, it is widely accepted that synthetic performance overestimates real performance, although to what degree this may happen remains unknown. This paper investigates the performance of 15 structure learning algorithms. We propose a methodology that applies the algorithms to data that incorporates synthetic noise, in an effort to better understand the performance of structure learning algorithms when applied to real data. Each algorithm is tested over multiple case studies, sample sizes, types of noise, and assessed with multiple evaluation criteria. This work involved approximately 10,000 graphs with a total structure learning runtime of seven months. It provides the first large-scale empirical validation of BN structure learning algorithms under different assumptions of data noise. The results suggest that traditional synthetic performance may overestimate real-world performance by anywhere between 10% and more than 50%. They also show that while score-based learning is generally superior to constraint-based learning, a higher fitting score does not necessarily imply a more accurate causal graph. To facilitate comparisons with future studies, we have made all data, raw results, graphs and BN models freely available online.