# FLARE-AI: Flaw Reporting for AI

arXiv: https://arxiv.org/abs/2606.31567 · HTML: https://mlsanity.com/paper/2606.31567/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 83 references that are in the map.
- First posted on arXiv 30 Jun 2026.

## Builds on (most cited first, 28 of 28)

- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [Ethical and social risks of harm from Language Models](https://mlsanity.com/paper/2112.04359/) (2021)
- [Adversarial NLI: A New Benchmark for Natural Language Understanding](https://mlsanity.com/paper/1910.14599/) (2019)
- [Stealing Part of a Production Language Model](https://mlsanity.com/paper/2403.06634/) (2024)
- [Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction](https://mlsanity.com/paper/2210.05791/) (2022)
- [Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database](https://mlsanity.com/paper/2011.08512/) (2020)
- [Outsider Oversight: Designing a Third Party Audit Ecosystem for AI Governance](https://mlsanity.com/paper/2206.04737/) (2022)
- [Interactive Model Cards: A Human-Centered Approach to Model Documentation](https://mlsanity.com/paper/2205.02894/) (2022)
- [A Safe Harbor for AI Evaluation and Red Teaming](https://mlsanity.com/paper/2403.04893/) (2024)
- [Open-Sourcing Highly Capable Foundation Models: An evaluation of risks, benefits, and alternative methods for pursuing open-source objectives](https://mlsanity.com/paper/2311.09227/) (2023)
- [Image Cropping on Twitter: Fairness Metrics, their Limitations, and the Importance of Representation, Design, and Agency](https://mlsanity.com/paper/2105.08667/) (2021)
- [A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms](https://mlsanity.com/paper/2407.01294/) (2024)
- [Deepfakes on Demand: the rise of accessible non-consensual deepfake image generators](https://mlsanity.com/paper/2505.03859/) (2025)
- [Acceptable Use Policies for Foundation Models](https://mlsanity.com/paper/2409.09041/) (2024)
- [The Responsible Foundation Model Development Cheatsheet: A Review of Tools & Resources](https://mlsanity.com/paper/2406.16746/) (2024)
- [Bound by the Bounty: Collaboratively Shaping Evaluation Processes for Queer AI Harms](https://mlsanity.com/paper/2307.10223/) (2023)
- [Coordinated Flaw Disclosure for AI: Beyond Security Vulnerabilities](https://mlsanity.com/paper/2402.07039/) (2024)
- [In-House Evaluation Is Not Enough: Towards Robust Third-Party Flaw Disclosure for General-Purpose AI](https://mlsanity.com/paper/2503.16861/) (2025)
- [To Err is AI : A Case Study Informing LLM Flaw Reporting Practices](https://mlsanity.com/paper/2410.12104/) (2024)
- [Bug Hunters' Perspectives on the Challenges and Benefits of the Bug Bounty Ecosystem](https://mlsanity.com/paper/2301.04781/) (2023)
- [Who Evaluates AI's Social Impacts? Mapping Coverage and Gaps in First and Third Party Evaluations](https://mlsanity.com/paper/2511.05613/) (2025)
- [Toward Comprehensive Risk Assessments and Assurance of AI-Based Systems](https://mlsanity.com/paper/2606.29390/) (2026)
- [Aggregated Individual Reporting for Post-Deployment Evaluation](https://mlsanity.com/paper/2506.18133/) (2025)
- [An External Fairness Evaluation of LinkedIn Talent Search](https://mlsanity.com/paper/2511.10752/) (2025)
- [Merchants of Vulnerabilities: How Bug Bounty Programs Benefit Software Vendors](https://mlsanity.com/paper/2404.17497/) (2024)
- [From Firewalls to Frontiers: AI Red-Teaming is a Domain-Specific Evolution of Cyber Red-Teaming](https://mlsanity.com/paper/2509.11398/) (2025)
- ["Abuse Risks are Often Inherent to Product Features": Exploring AI Vendors' Bug Bounty and Responsible Disclosure Policies](https://mlsanity.com/paper/2509.06136/) (2025)

## Abstract (arXiv)

Flaw reporting for deployed AI systems is fundamental to identifying system failures and improving AI safety. Yet the AI reporting ecosystem is fragmented: researchers who identify flaws often do not know what or where to report, and groups who receive reports rarely share them with other relevant stakeholders. As a result, good-faith reporters duplicate effort by submitting many different forms, and recipients lack standardized, triage-ready information. We audit 12 reporting systems published by AI developers, cybersecurity groups, and AI flaw aggregators, identifying five recurring design challenges spanning discoverability, scope, information collection, coordination, and guidance for strict-liability cases. Building on this analysis and feedback from 49 experts across 32 organizations representing developers, security researchers, and ecosystem coordinators, we introduce FLARE-AI, an open-source AI flaw reporting system designed for interoperability with existing systems. FLARE-AI streamlines flaw report creation by collecting triage-relevant information through conditional logic and early classification, then enables optional dissemination of standardized, machine-readable reports to multiple developers, coordinators, and incident registries from a single submission. By lowering barriers to reporting AI flaws and improving interoperability across stakeholders, FLARE-AI helps break down silos and accelerate remediation across the AI ecosystem.