# Addressing the regulatory gap: moving towards an EU AI audit ecosystem beyond the AI Act by including civil society

arXiv: https://arxiv.org/abs/2403.07904 · HTML: https://mlsanity.com/paper/2403.07904/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 101 references that are in the map.
- First posted on arXiv 26 Feb 2024.

## Builds on (most cited first, 20 of 20)

- [Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing](https://mlsanity.com/paper/2001.00973/) (2020)
- [Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction](https://mlsanity.com/paper/2210.05791/) (2022)
- [Auditing large language models: a three-layered approach](https://mlsanity.com/paper/2302.08500/) (2023)
- [Black-Box Access is Insufficient for Rigorous AI Audits](https://mlsanity.com/paper/2401.14446/) (2024)
- [Discrimination in Online Ad Delivery](https://mlsanity.com/paper/1301.6822/) (2013)
- [Outsider Oversight: Designing a Third Party Audit Ecosystem for AI Governance](https://mlsanity.com/paper/2206.04737/) (2022)
- [Auditing Radicalization Pathways on YouTube](https://mlsanity.com/paper/1908.08313/) (2019)
- [Who Audits the Auditors? Recommendations from a field scan of the algorithmic auditing ecosystem](https://mlsanity.com/paper/2310.02521/) (2023)
- [AI auditing: The Broken Bus on the Road to AI Accountability](https://mlsanity.com/paper/2401.14462/) (2024)
- [Ethics-Based Auditing to Develop Trustworthy AI](https://mlsanity.com/paper/2105.00002/) (2021)
- [Problematic Machine Behavior: A Systematic Literature Review of Algorithm Audits](https://mlsanity.com/paper/2102.04256/) (2021)
- [An Algorithmic Framework for Bias Bounties](https://mlsanity.com/paper/2201.10408/) (2022)
- [Conformity Assessments and Post-market Monitoring: A Guide to the Role of Auditing in the Proposed European AI Regulation](https://mlsanity.com/paper/2111.05071/) (2021)
- [Algorithms as Social-Ecological-Technological Systems: an Environmental Justice Lens on Algorithmic Audits](https://mlsanity.com/paper/2305.05733/) (2023)
- [Auditing Elon Musk's Impact on Hate Speech and Bots](https://mlsanity.com/paper/2304.04129/) (2023)
- [Ethically Aligned Design of Autonomous Systems: Industry viewpoint and an empirical study](https://mlsanity.com/paper/1906.07946/) (2019)
- [Non-portability of Algorithmic Fairness in India](https://mlsanity.com/paper/2012.03659/) (2020)
- [Participation and Division of Labor in User-Driven Algorithm Audits: How Do Everyday Users Work together to Surface Algorithmic Harms?](https://mlsanity.com/paper/2304.02134/) (2023)
- [The E.U.'s Artificial Intelligence Act: An Ordoliberal Assessment](https://mlsanity.com/paper/2309.12336/) (2023)
- [Audit, Don't Explain -- Recommendations Based on a Socio-Technical Understanding of ML-Based Systems](https://mlsanity.com/paper/2107.09917/) (2021)

## Built on by (most cited first, 7 of 7)

- [Lost in Moderation: How Commercial Content Moderation APIs Over- and Under-Moderate Group-Targeted Hate Speech and Linguistic Variations](https://mlsanity.com/paper/2503.01623/) (2025)
- [The Accountability Paradox: How Platform API Restrictions Undermine AI Transparency Mandates](https://mlsanity.com/paper/2505.11577/) (2025)
- [Audit Me If You Can: Query-Efficient Active Fairness Auditing of Black-Box LLMs](https://mlsanity.com/paper/2601.03087/) (2026)
- [Envisioning Stakeholder-Action Pairs to Mitigate Negative Impacts of AI: A Participatory Approach to Inform Policy Making](https://mlsanity.com/paper/2502.14869/) (2025)
- [Bye Bye Perspective API: Lessons for Building and Governing Measurement Infrastructure](https://mlsanity.com/paper/2604.25580/) (2026)
- [Informing AI Policy Assessment using Large-Scale Simulation of Interventions](https://mlsanity.com/paper/2605.27395/) (2026)
- [Scaling, Lock-In, and Proxy Compliance: A Political Economy of Responsible AI](https://mlsanity.com/paper/2607.28023/) (2026)

## Abstract (arXiv)

The European legislature has proposed the Digital Services Act (DSA) and Artificial Intelligence Act (AIA) to regulate platforms and Artificial Intelligence (AI) products. We review to what extent third-party audits are part of both laws and how is access to information on models and the data provided. By considering the value of third-party audits and third-party data access in an audit ecosystem, we identify a regulatory gap in that the AIA does not provide access to data for researchers and civil society. Our contributions to the literature include: (1) Defining an AI audit ecosystem incorporating compliance and oversight. (2) Highlighting a regulatory gap within the DSA and AIA regulatory framework, preventing the establishment of an AI audit ecosystem that has effective oversight by civil society and academia. (3) Emphasizing that third-party audits by research and civil society must be part of that ecosystem, we call for AIA amendments and delegated acts to include data and model access for certain AI products. Furthermore, we call for the DSA to provide NGOs and investigative journalists with data access to platforms by delegated acts and for adaptions and amendments of the AIA to provide third-party audits and data and model access, at least for high-risk systems. Regulations modeled after EU AI regulations should enable data access and third-party audits, fostering an AI audit ecosystem that promotes compliance and oversight mechanisms.