# Audit Cards: Contextualizing AI Evaluations

arXiv: https://arxiv.org/abs/2504.13839 · HTML: https://mlsanity.com/paper/2504.13839/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Apr 2025.

## Built on by (most cited first, 10 of 10)

- [STREAM (ChemBio): A Standard for Transparently Reporting Evaluations in AI Model Reports](https://mlsanity.com/paper/2508.09853/) (2025)
- [Computational Hermeneutics: Evaluating generative AI as a cultural technology](https://mlsanity.com/paper/2604.16403/) (2026)
- [Who Evaluates AI's Social Impacts? Mapping Coverage and Gaps in First and Third Party Evaluations](https://mlsanity.com/paper/2511.05613/) (2025)
- [Evaluation Cards: An Interpretive Layer for AI Evaluation Reporting](https://mlsanity.com/paper/2606.09809/) (2026)
- [Intelligent Automation for Embodied Benchmark Construction: Pipelines, Embodiments, Simulators, and Trends](https://mlsanity.com/paper/2606.12207/) (2026)
- [Applications of Risk Science to AI Fairness Evaluation: Principles, Challenges, and Best Practices](https://mlsanity.com/paper/2608.29478/) (2026)
- [A Conceptual Framework for AI Capability Evaluations](https://mlsanity.com/paper/2506.18213/) (2025)
- [Deprecating Benchmarks: Criteria and Framework](https://mlsanity.com/paper/2507.06434/) (2025)
- [AI Transparency: Governance Compliance or Stakeholder Requirements?](https://mlsanity.com/paper/2606.30652/) (2026)
- [When Is an Agent Evaluation Over? Outcome Finality and Cross-Unit Separation](https://mlsanity.com/paper/2608.14940/) (2026)

## Abstract (arXiv)

AI governance frameworks increasingly rely on audits, yet the results of their underlying evaluations require interpretation and context to be meaningfully informative. Even technically rigorous evaluations can offer little useful insight if reported selectively or obscurely. Current literature focuses primarily on technical best practices, but evaluations are an inherently sociotechnical process, and there is little guidance on reporting procedures and context. Through literature review, stakeholder interviews, and analysis of governance frameworks, we propose "audit cards" to make this context explicit. We identify six key types of contextual features to report and justify in audit cards: auditor identity, evaluation scope, methodology, resource access, process integrity, and review mechanisms. Through analysis of existing evaluation reports, we find significant variation in reporting practices, with most reports omitting crucial contextual information such as auditors' backgrounds, conflicts of interest, and the level and type of access to models. We also find that most existing regulations and frameworks lack guidance on rigorous reporting. In response to these shortcomings, we argue that audit cards can provide a structured format for reporting key claims alongside their justifications, enhancing transparency, facilitating proper interpretation, and establishing trust in reporting.