# Standardization of Post-Publication Code Verification by Journals is Possible with the Support of the Community

arXiv: https://arxiv.org/abs/2601.07189 · HTML: https://mlsanity.com/paper/2601.07189/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 34 references that are in the map.
- First posted on arXiv 12 Jan 2026.

## Builds on (most cited first, 9 of 9)

- [Deep Reinforcement Learning that Matters](https://mlsanity.com/paper/1709.06560/) (2017)
- [Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)](https://mlsanity.com/paper/2003.12206/) (2020)
- [Accounting for Variance in Machine Learning Benchmarks](https://mlsanity.com/paper/2103.03098/) (2021)
- [AxCell: Automatic Extraction of Results from Machine Learning Papers](https://mlsanity.com/paper/2004.14356/) (2020)
- [Tortured phrases: A dubious writing style emerging in science. Evidence of critical issues affecting established journals](https://mlsanity.com/paper/2107.06751/) (2021)
- [Revisiting the Necessity of Graph Learning and Common Graph Benchmarks](https://mlsanity.com/paper/2412.06173/) (2024)
- [On the (In)Security of Loading Machine Learning Models](https://mlsanity.com/paper/2509.06703/) (2025)
- [Improper legitimization of hijacked journals through citations](https://mlsanity.com/paper/2209.04703/) (2022)
- [Tackling paper mills requires us to prevent future contamination and clean up the past -- the case of the journal Bioengineered](https://mlsanity.com/paper/2503.21267/) (2025)

## Abstract (arXiv)

Reproducibility remains a challenge in machine learning research. While code and data availability requirements have become increasingly common, post-publication verification in journals is still limited and unformalized. This position paper argues that it is plausible for journals and conference proceedings to implement post-publication verification. We propose a modification to ACM pre-publication verification badges that allows independent researchers to submit post-publication code replications to the journal, leading to visible verification badges included in the article metadata. Each article may earn up to two badges, each linked to verified code in its corresponding public repository. We describe the motivation, related initiatives, a formal framework, the potential impact, possible limitations, and alternative views.