# Tuning structure learning algorithms with out-of-sample and resampling strategies

arXiv: https://arxiv.org/abs/2306.13932 · HTML: https://mlsanity.com/paper/2306.13932/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 54 references that are in the map.
- First posted on arXiv 24 Jun 2023.

## Builds on (most cited first, 12 of 12)

- [DAGs with NO TEARS: Continuous Optimization for Structure Learning](https://mlsanity.com/paper/1803.01422/) (2018)
- [Learning Bayesian Networks: The Combination of Knowledge and Statistical Data](https://mlsanity.com/paper/1302.6815/) (2013)
- [On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practice](https://mlsanity.com/paper/2007.15745/) (2020)
- [A survey of Bayesian Network structure learning](https://mlsanity.com/paper/2109.11415/) (2021)
- [Large-scale empirical validation of Bayesian Network structure learning algorithms with noisy data](https://mlsanity.com/paper/2005.09020/) (2020)
- [Efficient Sampling and Structure Learning of Bayesian Networks](https://mlsanity.com/paper/1803.07859/) (2018)
- [Bayesian structure learning and sampling of Bayesian networks with the R package BiDAG](https://mlsanity.com/paper/2105.00488/) (2021)
- [Comparison of Bayesian predictive methods for model selection](https://mlsanity.com/paper/1503.08650/) (2015)
- [Effective and efficient structure learning with pruning and model averaging strategies](https://mlsanity.com/paper/2112.00398/) (2021)
- [Open problems in causal structure learning: A case study of COVID-19 in the UK](https://mlsanity.com/paper/2305.03859/) (2023)
- [Learning Bayesian networks from demographic and health survey data](https://mlsanity.com/paper/1912.00715/) (2019)
- [Scaling up Greedy Causal Search for Continuous Variables](https://mlsanity.com/paper/1507.07749/) (2015)

## Built on by (most cited first, 1 of 1)

- [Robustness of Algorithms for Causal Structure Learning to Hyperparameter Choice](https://mlsanity.com/paper/2310.18212/) (2023)

## Abstract (arXiv)

One of the challenges practitioners face when applying structure learning algorithms to their data involves determining a set of hyperparameters; otherwise, a set of hyperparameter defaults is assumed. The optimal hyperparameter configuration often depends on multiple factors, including the size and density of the usually unknown underlying true graph, the sample size of the input data, and the structure learning algorithm. We propose a novel hyperparameter tuning method, called the Out-of-sample Tuning for Structure Learning (OTSL), that employs out-of-sample and resampling strategies to estimate the optimal hyperparameter configuration for structure learning, given the input data set and structure learning algorithm. Synthetic experiments show that employing OTSL as a means to tune the hyperparameters of hybrid and score-based structure learning algorithms leads to improvements in graphical accuracy compared to the state-of-the-art. We also illustrate the applicability of this approach to real datasets from different disciplines.