mlsanityData to 14 Oct 2026

To Err is AI : A Case Study Informing LLM Flaw Reporting Practices

arXiv 2410.12104 ↗First posted 15 Oct 2024cs.CY cs.LG cs.SEcited by 5

Not on the map. Why?

Abstract (arXiv)

In August of 2024, 495 hackers generated evaluations in an open-ended bug bounty targeting the Open Language Model (OLMo) from The Allen Institute for AI. A vendor panel staffed by representatives of OLMo's safety program adjudicated changes to OLMo's documentation and awarded cash bounties to participants who successfully demonstrated a need for public disclosure clarifying the intent, capacities, and hazards of model deployment. This paper presents a collection of lessons learned, illustrative of flaw reporting best practices intended to reduce the likelihood of incidents and produce safer large language models (LLMs). These include best practices for safety reporting processes, their artifacts, and safety program staffing.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 22 of its 39 references that are in the map. First posted on arXiv 15 Oct 2024.

Builds on

22 of 39 references.

References · 22

  1. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  2. 2018Datasheets for Datasetscited by 1,386
  3. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
  4. 2018Model Cards for Model Reportingcited by 1,080
  5. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 722
  6. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 629
  7. 2024OLMo: Accelerating the Science of Language Modelscited by 613
  8. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 560
  9. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 434
  10. 2023Low-Resource Languages Jailbreak GPT-4cited by 309
  11. 2023MART: Improving LLM Safety with Multi-round Automatic Red-Teamingcited by 163
  12. 2023Understanding Catastrophic Forgetting in Language Models via Implicit Inferencecited by 144
10 more
  1. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 126
  2. 2020Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Databasecited by 96
  3. 2024On the Societal Impact of Open Foundation Modelscited by 65
  4. 2023Opening up ChatGPT: Tracking openness, transparency, and accountability in instruction-tuned text generatorscited by 56
  5. 2024A Safe Harbor for AI Evaluation and Red Teamingcited by 47
  6. 2024Consent in Crisis: The Rapid Decline of the AI Data Commonscited by 46
  7. 2024AI auditing: The Broken Bus on the Road to AI Accountabilitycited by 43
  8. 2023Can large language models democratize access to dual-use biotechnology?cited by 35
  9. 2023ChatGPT as an Attack Tool: Stealthy Textual Backdoor Attack via Blackbox Generative Model Triggercited by 31
  10. 2024Coordinated Flaw Disclosure for AI: Beyond Security Vulnerabilitiescited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 2.

5 papers.

  1. 2025In-House Evaluation Is Not Enough: Towards Robust Third-Party Flaw Disclosure for General-Purpose AIcited by 7
  2. 2025Benchmarking LLM-Assisted Blue Teaming via Standardized Threat Huntingcited by 6
  3. 2025Malicious and Unintentional Disclosure Risks in Large Language Models for Code Generationcited by 0
  4. 2026FLARE-AI: Flaw Reporting for AIcited by 0
  5. 2026Mapping the RAG Landscape: A Four Axis Taxonomy of Efficiency, Defense, Interactivity, and Reasoningcited by 0