# Reproducibility of Machine Learning-Based Fault Detection and Diagnosis for HVAC Systems in Buildings: An Empirical Study

arXiv: https://arxiv.org/abs/2508.00880 · HTML: https://mlsanity.com/paper/2508.00880/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 82 references that are in the map.
- First posted on arXiv 23 Jul 2025.

## Builds on (most cited first, 10 of 10)

- [Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)](https://mlsanity.com/paper/2003.12206/) (2020)
- [Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches](https://mlsanity.com/paper/1907.06902/) (2019)
- [A Step Toward Quantifying Independently Reproducible Machine Learning Research](https://mlsanity.com/paper/1909.06674/) (2019)
- [Reproducibility in Machine Learning-based Research: Overview, Barriers and Drivers](https://mlsanity.com/paper/2406.14325/) (2024)
- [A large-scale study on research code quality and execution](https://mlsanity.com/paper/2103.12793/) (2021)
- [Position: Why We Must Rethink Empirical Research in Machine Learning](https://mlsanity.com/paper/2405.02200/) (2024)
- [The Effect of Third Party Implementations on Reproducibility](https://mlsanity.com/paper/2307.14956/) (2023)
- [Benchmarking Machine Learning Technologies for Software Defect Detection](https://mlsanity.com/paper/1506.07563/) (2015)
- [Investigating Reproducibility in Deep Learning-Based Software Fault Prediction](https://mlsanity.com/paper/2402.05645/) (2024)
- [Exploring Code Comprehension in Scientific Programming: Preliminary Insights from Research Scientists](https://mlsanity.com/paper/2501.10037/) (2025)

## Abstract (arXiv)

Reproducibility is a cornerstone of scientific research, enabling independent verification and validation of empirical findings. The topic gained prominence in fields such as psychology and medicine, where concerns about non - replicable results sparked ongoing discussions about research practices. In recent years, the fast-growing field of Machine Learning (ML) has become part of this discourse, as it faces similar concerns about transparency and reliability. Some reproducibility issues in ML research are shared with other fields, such as limited access to data and missing methodological details. In addition, ML introduces specific challenges, including inherent nondeterminism and computational constraints. While reproducibility issues are increasingly recognized by the ML community and its major conferences, less is known about how these challenges manifest in applied disciplines. This paper contributes to closing this gap by analyzing the transparency and reproducibility standards of ML applications in building energy systems. The results indicate that nearly all articles are not reproducible due to insufficient disclosure across key dimensions of reproducibility. 72% of the articles do not specify whether the dataset used is public, proprietary, or commercially available. Only two papers share a link to their code - one of which was broken. Two-thirds of the publications were authored exclusively by academic researchers, yet no significant differences in reproducibility were observed compared to publications with industry-affiliated authors. These findings highlight the need for targeted interventions, including reproducibility guidelines, training for researchers, and policies by journals and conferences that promote transparency and reproducibility.