SIEVE: Towards Verifiable Certification for Code-datasets
Not on the map. Why?
Abstract (arXiv)
Code agents and empirical software engineering rely on public code datasets, yet these datasets lack verifiable quality guarantees. Static 'dataset cards' inform, but they are neither auditable nor do they offer statistical guarantees, making it difficult to attest to dataset quality. Teams build isolated, ad-hoc cleaning pipelines. This fragments effort and raises cost. We present SIEVE, a community-driven framework. It turns per-property checks into Confidence Cards-machine-readable, verifiable certificates with anytime-valid statistical bounds. We outline a research plan to bring SIEVE to maturity, replacing narrative cards with anytime-verifiable certification. This shift is expected to lower quality-assurance costs and increase trust in code-datasets.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 9 of its 18 references that are in the map. First posted on arXiv 2 Oct 2025.
Builds on
9 of 18 references.
References · 9
- 2018Time-uniform, nonparametric, nonasymptotic confidence sequencescited by 229
- 2024Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMscited by 229
- 2021CodeNet: A Large-Scale AI for Code Dataset for Learning a Diversity of Coding Taskscited by 200
- 2019Safe Testingcited by 101
- 2018The Dataset Nutrition Label: A Framework To Drive Higher Data Quality Standardscited by 97
- 2024Navigating Dataset Documentations in AI: A Large-Scale Analysis of Dataset Cards on Hugging Facecited by 27
- 2024Blockchain for Large Language Model Security and Safety: A Holistic Surveycited by 8
- 2023Open Datasheets: Machine-readable Documentation for Open Datasets and Responsible AI Assessmentscited by 7
- 2024A Standardized Machine-readable Dataset Documentation Format for Responsible AIcited by 7
Built on by
Not cited yet.