Towards best practices in AGI safety and governance: A survey of expert opinion
Not on the map. Why?
Abstract (arXiv)
A number of leading AI companies, including OpenAI, Google DeepMind, and Anthropic, have the stated goal of building artificial general intelligence (AGI) - AI systems that achieve or exceed human performance across a wide range of cognitive tasks. In pursuing this goal, they may develop and deploy AI systems that pose particularly significant risks. While they have already taken some measures to mitigate these risks, best practices have not yet emerged. To support the identification of best practices, we sent a survey to 92 leading experts from AGI labs, academia, and civil society and received 51 responses. Participants were asked how much they agreed with 50 statements about what AGI labs should do. Our main finding is that participants, on average, agreed with all of them. Many statements received extremely high levels of agreement. For example, 98% of respondents somewhat or strongly agreed that AGI labs should conduct pre-deployment risk assessments, dangerous capabilities evaluations, third-party model audits, safety restrictions on model usage, and red teaming. Ultimately, our list of statements may serve as a helpful foundation for efforts to develop best practices, standards, and regulations for AGI labs.
Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 May 2023.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2023: 12; 2024: 7; 2025: 5; 2026: 3.
27 papers.
Sort:
By area:
- 2023An Overview of Catastrophic AI Riskscited by 214
- 2023Can LLM-Generated Misinformation Be Detected?cited by 173
- 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 123
- 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
- 2023Evil Geniuses: Delving into the Safety of LLM-based Agentscited by 103
- 2023Frontier AI Regulation: Managing Emerging Risks to Public Safetycited by 95
- 2024Red-Teaming for Generative AI: Silver Bullet or Security Theater?cited by 81
- 2023Managing extreme AI risks amid rapid progresscited by 48
- 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 44
- 2023Look Before You Leap: An Exploratory Study of Uncertainty Measurement for Large Language Modelscited by 40
- 2025Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilitiescited by 36
- 2023Open-Sourcing Highly Capable Foundation Models: An evaluation of risks, benefits, and alternative methods for pursuing open-source objectivescited by 31
15 more
- 2024Can Editing LLMs Inject Harm?cited by 22
- 2023From Google Gemini to OpenAI Q* (Q-Star): A Survey of Reshaping the Generative Artificial Intelligence (AI) Research Landscapecited by 18
- 2024Evaluating AI Evaluation: Perils and Prospectscited by 11
- 2024Responsible Reporting for Frontier AI Developmentcited by 10
- 2025STREAM (ChemBio): A Standard for Transparently Reporting Evaluations in AI Model Reportscited by 9
- 2025Thinking Beyond Tokens: From Brain-Inspired Intelligence to Cognitive Foundations for Artificial General Intelligence and its Societal Impactcited by 6
- 2023An International Consortium for Evaluations of Societal-Scale Risks from Advanced AIcited by 5
- 2023Towards Responsible Governance of Biological Design Toolscited by 5
- 2023The Chai Platform's AI Safety Frameworkcited by 1
- 2023Contextual Confidence and Generative AIcited by 1
- 2026Alignment Is Not Enough: A Relational Framework for Moral Standing in Human-AI Interactioncited by 1
- 2024Evaluating the Capabilities of LLMs for Supporting Anticipatory Impact Assessmentcited by 0
- 2025From Turing to Tomorrow: The UK's Approach to AI Regulationcited by 0
- 2026From Failure to Alignment: A Requirements Engineering Framework for Machine Learning Systemscited by 0
- 2026Silent Revision: Measuring Undisclosed Change in the Safety Frameworks of Frontier AI Developerscited by 0
- 2026Silent Revision: Measuring Undisclosed Change in the Safety Frameworks of Frontier AI Developers8 Sep 2026
- 2026From Failure to Alignment: A Requirements Engineering Framework for Machine Learning Systems30 Jun 2026
- 2026Alignment Is Not Enough: A Relational Framework for Moral Standing in Human-AI Interaction13 Feb 2026
- 2025STREAM (ChemBio): A Standard for Transparently Reporting Evaluations in AI Model Reports13 Aug 2025
- 2025From Turing to Tomorrow: The UK's Approach to AI Regulation3 Jul 2025
- 2025Thinking Beyond Tokens: From Brain-Inspired Intelligence to Cognitive Foundations for Artificial General Intelligence and its Societal Impact1 Jul 2025
- 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment22 Apr 2025
- 2025Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilities3 Feb 2025
- 2024Can Editing LLMs Inject Harm?29 Jul 2024
- 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)20 Jul 2024
- 2024Evaluating AI Evaluation: Perils and Prospects12 Jul 2024
- 2024Responsible Reporting for Frontier AI Development3 Apr 2024
15 more
- 2024Evaluating the Capabilities of LLMs for Supporting Anticipatory Impact Assessment31 Jan 2024
- 2024Red-Teaming for Generative AI: Silver Bullet or Security Theater?29 Jan 2024
- 2024Black-Box Access is Insufficient for Rigorous AI Audits25 Jan 2024
- 2023From Google Gemini to OpenAI Q* (Q-Star): A Survey of Reshaping the Generative Artificial Intelligence (AI) Research Landscape18 Dec 2023
- 2023Towards Responsible Governance of Biological Design Tools27 Nov 2023
- 2023Evil Geniuses: Delving into the Safety of LLM-based Agents20 Nov 2023
- 2023Contextual Confidence and Generative AI2 Nov 2023
- 2023Managing extreme AI risks amid rapid progress26 Oct 2023
- 2023An International Consortium for Evaluations of Societal-Scale Risks from Advanced AI22 Oct 2023
- 2023Open-Sourcing Highly Capable Foundation Models: An evaluation of risks, benefits, and alternative methods for pursuing open-source objectives29 Sep 2023
- 2023Can LLM-Generated Misinformation Be Detected?25 Sep 2023
- 2023Look Before You Leap: An Exploratory Study of Uncertainty Measurement for Large Language Models16 Jul 2023
- 2023Frontier AI Regulation: Managing Emerging Risks to Public Safety6 Jul 2023
- 2023An Overview of Catastrophic AI Risks21 Jun 2023
- 2023The Chai Platform's AI Safety Framework5 Jun 2023
- 2024Red-Teaming for Generative AI: Silver Bullet or Security Theater?cited by 81
- 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 44
- 2024Can Editing LLMs Inject Harm?cited by 22
- 2024Can Editing LLMs Inject Harm?29 Jul 2024
- 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)20 Jul 2024
- 2024Red-Teaming for Generative AI: Silver Bullet or Security Theater?29 Jan 2024
- 2023Evil Geniuses: Delving into the Safety of LLM-based Agentscited by 103
- 2023Evil Geniuses: Delving into the Safety of LLM-based Agents20 Nov 2023
- 2023An Overview of Catastrophic AI Riskscited by 214
- 2023Can LLM-Generated Misinformation Be Detected?cited by 173
- 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 123
- 2023Frontier AI Regulation: Managing Emerging Risks to Public Safetycited by 95
- 2023Managing extreme AI risks amid rapid progresscited by 48
- 2023Look Before You Leap: An Exploratory Study of Uncertainty Measurement for Large Language Modelscited by 40
- 2025Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilitiescited by 36
- 2023Open-Sourcing Highly Capable Foundation Models: An evaluation of risks, benefits, and alternative methods for pursuing open-source objectivescited by 31
- 2024Evaluating AI Evaluation: Perils and Prospectscited by 11
- 2024Responsible Reporting for Frontier AI Developmentcited by 10
- 2025STREAM (ChemBio): A Standard for Transparently Reporting Evaluations in AI Model Reportscited by 9
- 2023An International Consortium for Evaluations of Societal-Scale Risks from Advanced AIcited by 5
12 most cited of 20 All 20 (JSON)
- 2026Silent Revision: Measuring Undisclosed Change in the Safety Frameworks of Frontier AI Developers8 Sep 2026
- 2026From Failure to Alignment: A Requirements Engineering Framework for Machine Learning Systems30 Jun 2026
- 2026Alignment Is Not Enough: A Relational Framework for Moral Standing in Human-AI Interaction13 Feb 2026
- 2025STREAM (ChemBio): A Standard for Transparently Reporting Evaluations in AI Model Reports13 Aug 2025
- 2025From Turing to Tomorrow: The UK's Approach to AI Regulation3 Jul 2025
- 2025Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilities3 Feb 2025
- 2024Evaluating AI Evaluation: Perils and Prospects12 Jul 2024
- 2024Responsible Reporting for Frontier AI Development3 Apr 2024
- 2024Evaluating the Capabilities of LLMs for Supporting Anticipatory Impact Assessment31 Jan 2024
- 2024Black-Box Access is Insufficient for Rigorous AI Audits25 Jan 2024
- 2023Towards Responsible Governance of Biological Design Tools27 Nov 2023
- 2023Contextual Confidence and Generative AI2 Nov 2023
12 newest of 20 All 20 (JSON)