Position: Embracing Negative Results in Machine Learning
Not on the map. Why?
Abstract (arXiv)
Publications proposing novel machine learning methods are often primarily rated by exhibited predictive performance on selected problems. In this position paper we argue that predictive performance alone is not a good indicator for the worth of a publication. Using it as such even fosters problems like inefficiencies of the machine learning research community as a whole and setting wrong incentives for researchers. We therefore put out a call for the publication of "negative" results, which can help alleviate some of these problems and improve the scientific output of the machine learning research community. To substantiate our position, we present the advantages of publishing negative results and provide concrete measures for the community to move towards a paradigm where their publication is normalized.
Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 12 of its 27 references that are in the map. First posted on arXiv 6 Jun 2024.
Builds on
12 of 27 references.
References · 12
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
- 2014Explaining and Harnessing Adversarial Examplescited by 7,364
- 2013Intriguing properties of neural networkscited by 5,155
- 2014Going Deeper with Convolutionscited by 4,241
- 2018YOLOv3: An Incremental Improvementcited by 2,659
- 2020Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)cited by 322
- 2020Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scanscited by 158
- 2021Torch.manual_seed(3407) is all you need: On the influence of random seeds in deep learning architectures for computer visioncited by 81
- 2019NAS evaluation is frustratingly hardcited by 70
- 2018Re-evaluating Evaluationcited by 47
- 2022Predicting the Future of AI with AI: High-quality link prediction in an exponentially growing knowledge networkcited by 36
- 2023Pre-registration for Predictive Modelingcited by 13
Built on by
Citing papers per year: 2025: 2; 2026: 4.
6 papers.
- 2025Position: Causal Machine Learning Requires Rigorous Synthetic Experiments for Broader Adoptioncited by 6
- 2025Not All Explanations for Deep Learning Phenomena Are Equally Valuablecited by 3
- 2026Make Mechanistic Interpretability Auditable: A Call to Develop Guidelines via Continuous Collaborative Reviewingcited by 2
- 2026Form, Not Content? A Preregistered, Placebo-Controlled Evaluation of Learned Error-Conditioned Self-Repair Through Prompts and Weights in Frozen Small Code Modelscited by 2
- 2026MetaLead: A Comprehensive Human-Curated Leaderboard Dataset for Transparent Reporting of Machine Learning Experimentscited by 0
- 2026Data Diversity, Not Frequency Invariance: A Controlled and Self-Audited Study of Compression-Robust Deepfake Detectioncited by 0
- 2026Data Diversity, Not Frequency Invariance: A Controlled and Self-Audited Study of Compression-Robust Deepfake Detection26 Aug 2026
- 2026Form, Not Content? A Preregistered, Placebo-Controlled Evaluation of Learned Error-Conditioned Self-Repair Through Prompts and Weights in Frozen Small Code Models14 Jul 2026
- 2026Make Mechanistic Interpretability Auditable: A Call to Develop Guidelines via Continuous Collaborative Reviewing24 Apr 2026
- 2026MetaLead: A Comprehensive Human-Curated Leaderboard Dataset for Transparent Reporting of Machine Learning Experiments30 Jan 2026
- 2025Position: Causal Machine Learning Requires Rigorous Synthetic Experiments for Broader Adoption12 Aug 2025
- 2025Not All Explanations for Deep Learning Phenomena Are Equally Valuable29 Jun 2025