# Auto-BenchmarkCard: Automated Synthesis of Benchmark Documentation

arXiv: https://arxiv.org/abs/2512.09577 · HTML: https://mlsanity.com/paper/2512.09577/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 8 references that are in the map.
- First posted on arXiv 10 Dec 2025.

## Builds on (most cited first, 6 of 6)

- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [BetterBench: Assessing AI Benchmarks, Uncovering Issues, and Establishing Best Practices](https://mlsanity.com/paper/2411.12990/) (2024)
- [Docling: An Efficient Open-Source Toolkit for AI-driven Document Conversion](https://mlsanity.com/paper/2501.17887/) (2025)
- [Unitxt: Flexible, Shareable and Reusable Data Preparation and Evaluation for Generative AI](https://mlsanity.com/paper/2401.14019/) (2024)
- [RiskRAG: A Data-Driven Solution for Improved AI Model Risk Reporting](https://mlsanity.com/paper/2504.08952/) (2025)
- [AI Risk Atlas: Taxonomy and Tooling for Navigating AI Risks and Resources](https://mlsanity.com/paper/2503.05780/) (2025)

## Built on by (most cited first, 1 of 1)

- [Every Eval Ever: A Unifying Schema and Community Repository for AI Evaluation Results](https://mlsanity.com/paper/2606.14516/) (2026)

## Abstract (arXiv)

We present Auto-BenchmarkCard, a workflow for generating validated descriptions of AI benchmarks. Benchmark documentation is often incomplete or inconsistent, making it difficult to interpret and compare benchmarks across tasks or domains. Auto-BenchmarkCard addresses this gap by combining multi-agent data extraction from heterogeneous sources (e.g., Hugging Face, Unitxt, academic papers) with LLM-driven synthesis. A validation phase evaluates factual accuracy through atomic entailment scoring using the FactReasoner tool. This workflow has the potential to promote transparency, comparability, and reusability in AI benchmark reporting, enabling researchers and practitioners to better navigate and evaluate benchmark choices.