# Trust AI Regulation? Discerning users are vital to build trust and effective AI regulation arXiv: https://arxiv.org/abs/2403.09510 ยท HTML: https://mlsanity.com/paper/2403.09510/ ## Key facts - Cited by 9 papers in the map (data through 14 Oct 2026). - Builds on 24 of its 77 references that are in the map. - First posted on arXiv 14 Mar 2024. ## Builds on (most cited first, 24 of 24) - [Model evaluation for extreme risks](https://mlsanity.com/paper/2305.15324/) (2023) - [Frontier AI Regulation: Managing Emerging Risks to Public Safety](https://mlsanity.com/paper/2307.03718/) (2023) - [Computing Power and the Governance of Artificial Intelligence](https://mlsanity.com/paper/2402.08797/) (2024) - [The Role of Cooperation in Responsible AI Development](https://mlsanity.com/paper/1907.04534/) (2019) - [What does it take to catch a Chinchilla? Verifying Rules on Large-Scale Neural Network Training via Compute Monitoring](https://mlsanity.com/paper/2303.11341/) (2023) - [Risk assessment at AGI companies: A review of popular risk assessment techniques from other safety-critical industries](https://mlsanity.com/paper/2307.08823/) (2023) - [The Brussels Effect and Artificial Intelligence: How EU regulation will impact the global AI market](https://mlsanity.com/paper/2208.12645/) (2022) - [AI Certification: Advancing Ethical Practice by Reducing Information Asymmetries](https://mlsanity.com/paper/2105.10356/) (2021) - [When to (or not to) trust intelligent machines: Insights from an evolutionary game theory analysis of trust in repeated games](https://mlsanity.com/paper/2007.11338/) (2020) - [Why and How Governments Should Monitor AI Development](https://mlsanity.com/paper/2108.12427/) (2021) - [Regulatory Markets for AI Safety](https://mlsanity.com/paper/2001.00078/) (2019) - [Coordinated pausing: An evaluation-based coordination scheme for frontier AI developers](https://mlsanity.com/paper/2310.00374/) (2023) - [Voluntary safety commitments provide an escape from over-regulation in AI development](https://mlsanity.com/paper/2104.03741/) (2021) - [Mediating Artificial Intelligence Developments through Negative and Positive Incentives](https://mlsanity.com/paper/2010.00403/) (2020) - [Artificial Intelligence Development Races in Heterogeneous Settings](https://mlsanity.com/paper/2012.15234/) (2020) - [International Institutions for Advanced AI](https://mlsanity.com/paper/2307.04699/) (2023) - [Cost efficiency of institutional incentives in finite populations](https://mlsanity.com/paper/2103.01131/) (2021) - [International Governance of Civilian AI: A Jurisdictional Certification Approach](https://mlsanity.com/paper/2308.15514/) (2023) - [The Tragedy of the AI Commons](https://mlsanity.com/paper/2006.05203/) (2020) - [Institutional Incentives for the Evolution of Committed Cooperation: Ensuring Participation is as Important as Enhancing Compliance](https://mlsanity.com/paper/2110.13307/) (2021) - [Pathways to Good Healthcare Services and Patient Satisfaction: An Evolutionary Game Theoretical Approach](https://mlsanity.com/paper/1907.07132/) (2019) - [Who is leading in AI? An analysis of industry AI research](https://mlsanity.com/paper/2312.00043/) (2023) - [Optimization of institutional incentives for cooperation in structured populations](https://mlsanity.com/paper/2301.06472/) (2023) - [Both eyes open: Vigilant Incentives help Regulatory Markets improve AI Safety](https://mlsanity.com/paper/2303.03174/) (2023) ## Built on by (most cited first, 9 of 9) - [Media and responsible AI governance: a game-theoretic and LLM analysis](https://mlsanity.com/paper/2503.09858/) (2025) - [Do LLMs trust AI regulation? Emerging behaviour of game-theoretic LLM agents](https://mlsanity.com/paper/2504.08640/) (2025) - [Social physics in the age of artificial intelligence](https://mlsanity.com/paper/2603.16900/) (2026) - [Can LLMs effectively provide game-theoretic-based scenarios for cybersecurity?](https://mlsanity.com/paper/2508.05670/) (2025) - [Can Media Act as a Soft Regulator of Safe AI Development? A Game Theoretical Analysis](https://mlsanity.com/paper/2509.02650/) (2025) - [Trust or Check? Understanding the (Evolutionary) Dynamics of User Trust in AI Systems](https://mlsanity.com/paper/2603.24742/) (2026) - [Mathematical Modelling of Ethical AI Use in Higher Education: A Coordination Game Framework for Future-Facing Learning](https://mlsanity.com/paper/2605.27400/) (2026) - [Social welfare optimisation under institutional reward and punishment](https://mlsanity.com/paper/2605.31330/) (2026) - [The Two Genie Game: Adoption and Welfare in Audit-Grounded AI Governance](https://mlsanity.com/paper/2606.28710/) (2026) ## Abstract (arXiv) There is general agreement that some form of regulation is necessary both for AI creators to be incentivised to develop trustworthy systems, and for users to actually trust those systems. But there is much debate about what form these regulations should take and how they should be implemented. Most work in this area has been qualitative, and has not been able to make formal predictions. Here, we propose that evolutionary game theory can be used to quantitatively model the dilemmas faced by users, AI creators, and regulators, and provide insights into the possible effects of different regulatory regimes. We show that creating trustworthy AI and user trust requires regulators to be incentivised to regulate effectively. We demonstrate the effectiveness of two mechanisms that can achieve this. The first is where governments can recognise and reward regulators that do a good job. In that case, if the AI system is not too risky for users then some level of trustworthy development and user trust evolves. We then consider an alternative solution, where users can condition their trust decision on the effectiveness of the regulators. This leads to effective regulation, and consequently the development of trustworthy AI and user trust, provided that the cost of implementing regulations is not too high. Our findings highlight the importance of considering the effect of different regulatory regimes from an evolutionary game theoretic perspective.