FLARE-AI: Flaw Reporting for AI
Not on the map. Why?
Abstract (arXiv)
Flaw reporting for deployed AI systems is fundamental to identifying system failures and improving AI safety. Yet the AI reporting ecosystem is fragmented: researchers who identify flaws often do not know what or where to report, and groups who receive reports rarely share them with other relevant stakeholders. As a result, good-faith reporters duplicate effort by submitting many different forms, and recipients lack standardized, triage-ready information. We audit 12 reporting systems published by AI developers, cybersecurity groups, and AI flaw aggregators, identifying five recurring design challenges spanning discoverability, scope, information collection, coordination, and guidance for strict-liability cases. Building on this analysis and feedback from 49 experts across 32 organizations representing developers, security researchers, and ecosystem coordinators, we introduce FLARE-AI, an open-source AI flaw reporting system designed for interoperability with existing systems. FLARE-AI streamlines flaw report creation by collecting triage-relevant information through conditional logic and early classification, then enables optional dissemination of standardized, machine-readable reports to multiple developers, coordinators, and incident registries from a single submission. By lowering barriers to reporting AI flaws and improving interoperability across stakeholders, FLARE-AI helps break down silos and accelerate remediation across the AI ecosystem.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 28 of its 83 references that are in the map. First posted on arXiv 30 Jun 2026.
Builds on
28 of 83 references.
References · 28
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
- 2018Model Cards for Model Reportingcited by 1,080
- 2021Ethical and social risks of harm from Language Modelscited by 966
- 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 807
- 2024Stealing Part of a Production Language Modelcited by 159
- 2022Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reductioncited by 138
- 2020Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Databasecited by 96
- 2022Outsider Oversight: Designing a Third Party Audit Ecosystem for AI Governancecited by 67
- 2022Interactive Model Cards: A Human-Centered Approach to Model Documentationcited by 61
- 2024A Safe Harbor for AI Evaluation and Red Teamingcited by 47
- 2023Open-Sourcing Highly Capable Foundation Models: An evaluation of risks, benefits, and alternative methods for pursuing open-source objectivescited by 31
- 2021Image Cropping on Twitter: Fairness Metrics, their Limitations, and the Importance of Representation, Design, and Agencycited by 22
16 more
- 2024A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harmscited by 19
- 2025Deepfakes on Demand: the rise of accessible non-consensual deepfake image generatorscited by 15
- 2024Acceptable Use Policies for Foundation Modelscited by 12
- 2024The Responsible Foundation Model Development Cheatsheet: A Review of Tools & Resourcescited by 11
- 2023Bound by the Bounty: Collaboratively Shaping Evaluation Processes for Queer AI Harmscited by 10
- 2024Coordinated Flaw Disclosure for AI: Beyond Security Vulnerabilitiescited by 8
- 2025In-House Evaluation Is Not Enough: Towards Robust Third-Party Flaw Disclosure for General-Purpose AIcited by 7
- 2024To Err is AI : A Case Study Informing LLM Flaw Reporting Practicescited by 5
- 2023Bug Hunters' Perspectives on the Challenges and Benefits of the Bug Bounty Ecosystemcited by 4
- 2025Who Evaluates AI's Social Impacts? Mapping Coverage and Gaps in First and Third Party Evaluationscited by 4
- 2026Toward Comprehensive Risk Assessments and Assurance of AI-Based Systemscited by 4
- 2025Aggregated Individual Reporting for Post-Deployment Evaluationcited by 3
- 2025An External Fairness Evaluation of LinkedIn Talent Searchcited by 3
- 2024Merchants of Vulnerabilities: How Bug Bounty Programs Benefit Software Vendorscited by 2
- 2025From Firewalls to Frontiers: AI Red-Teaming is a Domain-Specific Evolution of Cyber Red-Teamingcited by 2
- 2025"Abuse Risks are Often Inherent to Product Features": Exploring AI Vendors' Bug Bounty and Responsible Disclosure Policiescited by 1
Built on by
Not cited yet.