mlsanityData to 14 Oct 2026

STREAM (ChemBio): A Standard for Transparently Reporting Evaluations in AI Model Reports

arXiv 2508.09853 ↗First posted 13 Aug 2025cs.CY cs.AIcited by 10

Not on the map. Why?

Abstract (arXiv)

Evaluations of dangerous AI capabilities are important for managing catastrophic risks. Public transparency into these evaluations - including what they test, how they are conducted, and how their results inform decisions - is crucial for building trust in AI development. We propose STREAM (A Standard for Transparently Reporting Evaluations in AI Model Reports), a standard to improve how model reports disclose evaluation results, initially focusing on chemical and biological (ChemBio) benchmarks. Developed in consultation with 23 experts across government, civil society, academia, and frontier AI companies, this standard is designed to (1) be a practical resource to help AI developers present evaluation results more clearly, and (2) help third parties identify whether model reports provide sufficient detail to assess the rigor of the ChemBio evaluations. We concretely demonstrate our proposed best practices with "gold standard" examples, and also provide a three-page reporting template to enable AI developers to implement our recommendations more easily.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 50 of its 144 references that are in the map. First posted on arXiv 13 Aug 2025.

Builds on

50 of 144 references.

References · 50

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  3. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  4. 2018Model Cards for Model Reportingcited by 1,080
  5. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
  6. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 759
  7. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 686
  8. 2020Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)cited by 322
  9. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modificationscited by 231
  10. 2021What Will it Take to Fix Benchmarking in Natural Language Understanding?cited by 203
  11. 2024Foundational Challenges in Assuring Alignment and Safety of Large Language Modelscited by 199
  12. 2024Are We Done with MMLU?cited by 194
38 more
  1. 2018The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigationcited by 165
  2. 2024LAB-Bench: Measuring Capabilities of Language Models for Biology Researchcited by 156
  3. 2023Benchmarking Cognitive Biases in Large Language Models as Evaluatorscited by 150
  4. 2024Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluationscited by 148
  5. 2024An Adversarial Perspective on Machine Unlearning for AI Safetycited by 123
  6. 2023Mechanistically analyzing the effects of fine-tuning on procedurally defined taskscited by 115
  7. 2025Establishing Best Practices for Building Rigorous Agentic Benchmarkscited by 102
  8. 2023LongEval: Guidelines for Human Evaluation of Faithfulness in Long-form Summarizationcited by 96
  9. 2022Shortcut Learning of Large Language Models in Natural Language Understandingcited by 85
  10. 2024Artifacts or Abduction: How Do LLMs Answer Multiple-Choice Questions Without the Question?cited by 71
  11. 2023Style Over Substance: Evaluation Biases for Large Language Modelscited by 70
  12. 2024Can multiple-choice questions really be useful in detecting the abilities of LLMs?cited by 69
  13. 2024On the Societal Impact of Open Foundation Modelscited by 65
  14. 2020Towards falsifiable interpretability researchcited by 53
  15. 2024Safety Cases: How to Justify the Safety of Advanced AI Systemscited by 48
  16. 2025Toward an Evaluation Science for Generative AI Systemscited by 45
  17. 2023Ecosystem Graphs: The Social Footprint of Foundation Modelscited by 32
  18. 2025Virology Capabilities Test (VCT): A Multimodal Virology Q&A Benchmarkcited by 31
  19. 2023AI capabilities can be significantly improved without expensive retrainingcited by 28
  20. 2023Towards best practices in AGI safety and governance: A survey of expert opinioncited by 27
  21. 2023What's the Meaning of Superhuman Performance in Today's NLU?cited by 23
  22. 2025An Empirical Analysis of Uncertainty in Large Language Model Evaluationscited by 20
  23. 2024Holistic Safety and Responsibility Evaluations of Advanced AI Modelscited by 19
  24. 2024Position: Why We Must Rethink Empirical Research in Machine Learningcited by 19
  25. 2024Safety case template for frontier AI: A cyber inability argumentcited by 19
  26. 2024Safety cases for frontier AIcited by 18
  27. 2019The Scientific Method in the Science of Machine Learningcited by 15
  28. 2022System Cards for AI-Based Decision-Making for Public Policycited by 13
  29. 2024AI Cards: Towards an Applied Framework for Machine-Readable AI and Risk Documentation Inspired by the EU AI Actcited by 10
  30. 2025Audit Cards: Contextualizing AI Evaluationscited by 10
  31. 2025LLMs Outperform Experts on Challenging Biology Benchmarkscited by 9
  32. 2025Safety Cases: A Scalable Approach to Frontier AI Safetycited by 8
  33. 2025Skewed Score: A statistical framework to assess autograderscited by 7
  34. 2025Recommendations and Reporting Checklist for Rigorous & Transparent Human Baselines in Model Evaluationscited by 5
  35. 2025Preliminary suggestions for rigorous GPAI model evaluationscited by 5
  36. 2023AI Risk Profiles: A Standards Proposal for Pre-Deployment AI Risk Disclosurescited by 4
  37. 2025AI Companies Should Report Pre- and Post-Mitigation Safety Evaluationscited by 3
  38. 2025Asymmetry by Design: Boosting Cyber Defenders with Differential Access to AIcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 9.

10 papers.

  1. 2025The 2025 Foundation Model Transparency Indexcited by 11
  2. 2026Autonomous Research Agents: A Survey of AI Scientists and the Verification Gapcited by 9
  3. 2026Evaluation Cards: An Interpretive Layer for AI Evaluation Reportingcited by 4
  4. 2026Measuring Biological Capabilities and Risks of AI Agentscited by 1
  5. 2026RCTs for Frontier AI Governance: Methodological Challenges and Solutions for Human Uplift Studiescited by 0
  6. 2026Risk Reporting for Developers' Internal AI Model Usecited by 0
  7. 2026NeurIPS Should Require Reproducibility Standards for Frontier AI Safety Claimscited by 0
  8. 2026A Threshold Exceedance Framework for CBRN Uplift Evaluation in Frontier Language Modelscited by 0
  9. 2026Epistemic Norms for AI Safety and Alignment Researchcited by 0
  10. 2026Benchmark Contamination: A Taxonomy Organized by Defeated Mitigationcited by 0