# Attestable Audits: Verifiable AI Safety Benchmarks Using Trusted Execution Environments

arXiv: https://arxiv.org/abs/2506.23706 · HTML: https://mlsanity.com/paper/2506.23706/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 59 references that are in the map.
- First posted on arXiv 30 Jun 2025.

## Builds on (most cited first, 25 of 25)

- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions](https://mlsanity.com/paper/1905.10044/) (2019)
- [RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models](https://mlsanity.com/paper/2009.11462/) (2020)
- [Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference](https://mlsanity.com/paper/2403.04132/) (2024)
- [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015)
- [BBQ: A Hand-Built Bias Benchmark for Question Answering](https://mlsanity.com/paper/2110.08193/) (2021)
- [The NarrativeQA Reading Comprehension Challenge](https://mlsanity.com/paper/1712.07040/) (2017)
- [Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing](https://mlsanity.com/paper/2001.00973/) (2020)
- [Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims](https://mlsanity.com/paper/2004.07213/) (2020)
- [Stealing Part of a Production Language Model](https://mlsanity.com/paper/2403.06634/) (2024)
- [AI Sandbagging: Language Models can Strategically Underperform on Evaluations](https://mlsanity.com/paper/2406.07358/) (2024)
- [Auditing large language models: a three-layered approach](https://mlsanity.com/paper/2302.08500/) (2023)
- [Harms from Increasingly Agentic Algorithmic Systems](https://mlsanity.com/paper/2302.10329/) (2023)
- [Black-Box Access is Insufficient for Rigorous AI Audits](https://mlsanity.com/paper/2401.14446/) (2024)
- [Generalization or Memorization: Data Contamination and Trustworthy Evaluation for Large Language Models](https://mlsanity.com/paper/2402.15938/) (2024)
- [PPFL: Privacy-preserving Federated Learning with Trusted Execution Environments](https://mlsanity.com/paper/2104.14380/) (2021)
- [The Gradient of Generative AI Release: Methods and Considerations](https://mlsanity.com/paper/2302.04844/) (2023)
- [COMPL-AI Framework: A Technical Interpretation and LLM Benchmarking Suite for the EU Artificial Intelligence Act](https://mlsanity.com/paper/2410.07959/) (2024)
- [Machine Learning with Confidential Computing: A Systematization of Knowledge](https://mlsanity.com/paper/2208.10134/) (2022)
- [Auditing of AI: Legal, Ethical and Technical Approaches](https://mlsanity.com/paper/2407.06235/) (2024)
- [Actionable Guidance for High-Consequence AI Risk Management: Towards Standards Addressing AI Catastrophic Risks](https://mlsanity.com/paper/2206.08966/) (2022)
- [Private LoRA Fine-tuning of Open-Source LLMs with Homomorphic Encryption](https://mlsanity.com/paper/2505.07329/) (2025)
- [Heckler: Breaking Confidential VMs with Malicious Interrupts](https://mlsanity.com/paper/2404.03387/) (2024)
- [Trusted Machine Learning Models Unlock Private Inference for Problems Currently Infeasible with Cryptography](https://mlsanity.com/paper/2501.08970/) (2025)

## Built on by (most cited first, 8 of 8)

- [Proof-of-Guardrail in AI Agents and What (Not) to Trust from It](https://mlsanity.com/paper/2603.05786/) (2026)
- [Position: Behavioural Assurance Cannot Verify the Safety Claims Governance Now Demands](https://mlsanity.com/paper/2605.15164/) (2026)
- [Evidence-Grounded Verified Agentic Reasoning: A Path Toward Eliminating LLM Hallucination in Empirical Inference via Tool-Attested Kernel Proofs](https://mlsanity.com/paper/2607.12650/) (2026)
- [TRUST: A Decentralized Framework for Auditing Large Language Model Reasoning](https://mlsanity.com/paper/2510.20188/) (2025)
- [Computational Compliance for AI Regulation: Blueprint for a New Research Domain](https://mlsanity.com/paper/2601.04474/) (2026)
- [Making AI-Assisted Grant Evaluation Auditable without Exposing the Model](https://mlsanity.com/paper/2604.25200/) (2026)
- [Privacy-Preserving AI Verification via Minimal Information Disclosure](https://mlsanity.com/paper/2608.02774/) (2026)
- [SoK: Decentralized Agent Economic Infrastructure](https://mlsanity.com/paper/2610.01756/) (2026)

## Abstract (arXiv)

Benchmarks are important measures to evaluate safety and compliance of AI models at scale. However, they typically do not offer verifiable results and lack confidentiality for model IP and benchmark datasets. We propose Attestable Audits, which run inside Trusted Execution Environments and enable users to verify interaction with a compliant AI model. Our work protects sensitive data even when model provider and auditor do not trust each other. This addresses verification challenges raised in recent AI governance frameworks. We build a prototype demonstrating feasibility on typical audit benchmarks against Llama-3.1.