Trust AI Regulation? Discerning users are vital to build trust and effective AI regulation
Not on the map. Why?
Abstract (arXiv)
There is general agreement that some form of regulation is necessary both for AI creators to be incentivised to develop trustworthy systems, and for users to actually trust those systems. But there is much debate about what form these regulations should take and how they should be implemented. Most work in this area has been qualitative, and has not been able to make formal predictions. Here, we propose that evolutionary game theory can be used to quantitatively model the dilemmas faced by users, AI creators, and regulators, and provide insights into the possible effects of different regulatory regimes. We show that creating trustworthy AI and user trust requires regulators to be incentivised to regulate effectively. We demonstrate the effectiveness of two mechanisms that can achieve this. The first is where governments can recognise and reward regulators that do a good job. In that case, if the AI system is not too risky for users then some level of trustworthy development and user trust evolves. We then consider an alternative solution, where users can condition their trust decision on the effectiveness of the regulators. This leads to effective regulation, and consequently the development of trustworthy AI and user trust, provided that the cost of implementing regulations is not too high. Our findings highlight the importance of considering the effect of different regulatory regimes from an evolutionary game theoretic perspective.
Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 24 of its 77 references that are in the map. First posted on arXiv 14 Mar 2024.
Builds on
24 of 77 references.
References · 24
- 2023Model evaluation for extreme riskscited by 167
- 2023Frontier AI Regulation: Managing Emerging Risks to Public Safetycited by 95
- 2024Computing Power and the Governance of Artificial Intelligencecited by 38
- 2019The Role of Cooperation in Responsible AI Developmentcited by 21
- 2023What does it take to catch a Chinchilla? Verifying Rules on Large-Scale Neural Network Training via Compute Monitoringcited by 20
- 2023Risk assessment at AGI companies: A review of popular risk assessment techniques from other safety-critical industriescited by 19
- 2022The Brussels Effect and Artificial Intelligence: How EU regulation will impact the global AI marketcited by 18
- 2021AI Certification: Advancing Ethical Practice by Reducing Information Asymmetriescited by 15
- 2020When to (or not to) trust intelligent machines: Insights from an evolutionary game theory analysis of trust in repeated gamescited by 14
- 2019Regulatory Markets for AI Safetycited by 13
- 2021Why and How Governments Should Monitor AI Developmentcited by 13
- 2023Coordinated pausing: An evaluation-based coordination scheme for frontier AI developerscited by 13
12 more
- 2021Voluntary safety commitments provide an escape from over-regulation in AI developmentcited by 10
- 2020Mediating Artificial Intelligence Developments through Negative and Positive Incentivescited by 9
- 2020Artificial Intelligence Development Races in Heterogeneous Settingscited by 8
- 2021Cost efficiency of institutional incentives in finite populationscited by 8
- 2023International Institutions for Advanced AIcited by 8
- 2023International Governance of Civilian AI: A Jurisdictional Certification Approachcited by 8
- 2020The Tragedy of the AI Commonscited by 7
- 2021Institutional Incentives for the Evolution of Committed Cooperation: Ensuring Participation is as Important as Enhancing Compliancecited by 6
- 2019Pathways to Good Healthcare Services and Patient Satisfaction: An Evolutionary Game Theoretical Approachcited by 4
- 2023Who is leading in AI? An analysis of industry AI researchcited by 4
- 2023Optimization of institutional incentives for cooperation in structured populationscited by 2
- 2023Both eyes open: Vigilant Incentives help Regulatory Markets improve AI Safetycited by 2
Built on by
Citing papers per year: 2025: 4; 2026: 5.
9 papers.
- 2025Media and responsible AI governance: a game-theoretic and LLM analysiscited by 11
- 2025Do LLMs trust AI regulation? Emerging behaviour of game-theoretic LLM agentscited by 11
- 2026Social physics in the age of artificial intelligencecited by 4
- 2025Can Media Act as a Soft Regulator of Safe AI Development? A Game Theoretical Analysiscited by 3
- 2025Can LLMs effectively provide game-theoretic-based scenarios for cybersecurity?cited by 2
- 2026Trust or Check? Understanding the (Evolutionary) Dynamics of User Trust in AI Systemscited by 1
- 2026Mathematical Modelling of Ethical AI Use in Higher Education: A Coordination Game Framework for Future-Facing Learningcited by 0
- 2026Social welfare optimisation under institutional reward and punishmentcited by 0
- 2026The Two Genie Game: Adoption and Welfare in Audit-Grounded AI Governancecited by 0
- 2026The Two Genie Game: Adoption and Welfare in Audit-Grounded AI Governance27 Jun 2026
- 2026Social welfare optimisation under institutional reward and punishment29 May 2026
- 2026Mathematical Modelling of Ethical AI Use in Higher Education: A Coordination Game Framework for Future-Facing Learning23 Apr 2026
- 2026Trust or Check? Understanding the (Evolutionary) Dynamics of User Trust in AI Systems25 Mar 2026
- 2026Social physics in the age of artificial intelligence4 Mar 2026
- 2025Can Media Act as a Soft Regulator of Safe AI Development? A Game Theoretical Analysis2 Sep 2025
- 2025Can LLMs effectively provide game-theoretic-based scenarios for cybersecurity?4 Aug 2025
- 2025Do LLMs trust AI regulation? Emerging behaviour of game-theoretic LLM agents11 Apr 2025
- 2025Media and responsible AI governance: a game-theoretic and LLM analysis12 Mar 2025