mlsanityData to 7 Oct 2026

Frontier AI Regulation: Managing Emerging Risks to Public Safety

arXiv 2307.03718 ↗First posted 6 Jul 2023cs.CY cs.AIcited by 95

Not on the map. Why?

Abstract (arXiv)

Advanced AI models hold the promise of tremendous benefits for humanity, but society needs to proactively manage the accompanying risks. In this paper, we focus on what we term "frontier AI" models: highly capable foundation models that could possess dangerous capabilities sufficient to pose severe risks to public safety. Frontier AI models pose a distinct regulatory challenge: dangerous capabilities can arise unexpectedly; it is difficult to robustly prevent a deployed model from being misused; and, it is difficult to stop a model's capabilities from proliferating broadly. To address these challenges, at least three building blocks for the regulation of frontier models are needed: (1) standard-setting processes to identify appropriate requirements for frontier AI developers, (2) registration and reporting requirements to provide regulators with visibility into frontier AI development processes, and (3) mechanisms to ensure compliance with safety standards for the development and deployment of frontier AI models. Industry self-regulation is an important first step. However, wider societal discussions and government intervention will be needed to create standards and to ensure compliance with them. We consider several options to this end, including granting enforcement powers to supervisory authorities and licensure regimes for frontier AI models. Finally, we propose an initial set of safety standards. These include conducting pre-deployment risk assessments; external scrutiny of model behavior; using risk assessments to inform deployment decisions; and monitoring and responding to new information about model capabilities and uses post-deployment. We hope this discussion contributes to the broader conversation on how to balance public safety risks and innovation benefits from advances at the frontier of AI development.

Key facts. Cited by 95 papers in the map (data through 7 Oct 2026). Builds on 96 of its 249 references that are in the map. First posted on arXiv 6 Jul 2023.

Builds on

96 of 249 references.

References · 96

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2020Scaling Laws for Neural Language Modelscited by 6,411
  5. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  7. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  8. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  9. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  10. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  11. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  12. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
38 more
  1. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  2. 2023A Survey of Large Language Modelscited by 2,537
  3. 2022Emergent Abilities of Large Language Modelscited by 2,483
  4. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  5. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  6. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  7. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  8. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  9. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  10. 2016Concrete Problems in AI Safetycited by 1,582
  11. 2018Datasheets for Datasetscited by 1,371
  12. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
  13. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  14. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  15. 2018Model Cards for Model Reportingcited by 1,068
  16. 2022Scaling Laws for Reward Model Overoptimizationcited by 1,056
  17. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  18. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  19. 2022Red Teaming Language Models with Language Modelscited by 983
  20. 2022In-context Learning and Induction Headscited by 930
  21. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
  22. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  23. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  24. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 630
  25. 2017Deep Learning Scaling is Predictable, Empiricallycited by 556
  26. 2022Beyond neural scaling laws: beating power law scaling via data pruningcited by 519
  27. 2023Are Emergent Abilities of Large Language Models a Mirage?cited by 515
  28. 2020Scaling Laws for Autoregressive Generative Modelingcited by 510
  29. 2019Release Strategies and the Social Impacts of Language Modelscited by 505
  30. 2018AI safety via debatecited by 384
  31. 2020Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditingcited by 380
  32. 2020Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stoppingcited by 377
  33. 2023Large Language Models as Tool Makerscited by 293
  34. 2021Unsolved Problems in ML Safetycited by 291
  35. 2019Tackling Climate Change with Machine Learningcited by 264
  36. 2023The False Promise of Imitating Proprietary LLMscited by 254
  37. 2019Risks from Learned Optimization in Advanced Machine Learning Systemscited by 250
  38. 2022The Alignment Problem from a Deep Learning Perspectivecited by 246

50 most cited of 96 All 96 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 17; 2024: 36; 2025: 24; 2026: 18.

95 papers.

Sort:

By area:

  1. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 743
  2. 2023Evaluating Large Language Models: A Comprehensive Surveycited by 200
  3. 2023Can LLM-Generated Misinformation Be Detected?cited by 173
  4. 2024The Emerged Security and Privacy of LLM Agent: A Survey with Case Studiescited by 127
  5. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 123
  6. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  7. 2024Best Practices and Lessons Learned on Synthetic Datacited by 104
  8. 2024Visibility into AI Agentscited by 77
  9. 2024ToolSword: Unveiling Safety Issues of Large Language Models in Tool Learning Across Three Stagescited by 77
  10. 2024AIR-Bench 2024: A Safety Benchmark Based on Risk Categories from Regulations and Policiescited by 69
  11. 2024On the Societal Impact of Open Foundation Modelscited by 64
  12. 2023Managing extreme AI risks amid rapid progresscited by 48
38 more
  1. 2023Welfare Diplomacy: Benchmarking Language Model Cooperationcited by 43
  2. 2023Look Before You Leap: An Exploratory Study of Uncertainty Measurement for Large Language Modelscited by 40
  3. 2025AutoRedTeamer: Autonomous Red Teaming with Lifelong Attack Integrationcited by 38
  4. 2024Are large language models superhuman chemists?cited by 34
  5. 2023Open-Sourcing Highly Capable Foundation Models: An evaluation of risks, benefits, and alternative methods for pursuing open-source objectivescited by 31
  6. 2024Risks from Language Models for Automated Mental Healthcare: Ethics and Structure for Implementationcited by 27
  7. 2025Integrating LLMs with ITS: Recent Advances, Potentials, Challenges, and Future Directionscited by 23
  8. 2024Can Editing LLMs Inject Harm?cited by 22
  9. 2024Towards Trustworthy AI: A Review of Ethical and Robust Large Language Modelscited by 21
  10. 2025Frontier AI's Impact on the Cybersecurity Landscapecited by 21
  11. 2023Position: Key Claims in LLM Research Have a Long Tail of Footnotescited by 20
  12. 2025FORTRESS: Frontier Risk Evaluation for National Security and Public Safetycited by 20
  13. 2024Mapping the Ethics of Generative AI: A Comprehensive Scoping Reviewcited by 19
  14. 2023Market Concentration Implications of Foundation Modelscited by 15
  15. 2024On Catastrophic Inheritance of Large Foundation Modelscited by 15
  16. 2024Foundation Model Transparency Reportscited by 14
  17. 2024Training Compute Thresholds: Features and Functions in AI Regulationcited by 13
  18. 2024Swiss Cheese Model for AI Safety: A Taxonomy and Reference Architecture for Multi-Layered Guardrails of Foundation Model Based Agentscited by 13
  19. 2023A Systematic Survey of Prompt Engineering on Vision-Language Foundation Modelscited by 12
  20. 2023Towards Publicly Accountable Frontier LLMs: Building an External Scrutiny Ecosystem under the ASPIRE Frameworkcited by 11
  21. 2024Evaluating AI Evaluation: Perils and Prospectscited by 11
  22. 2025Do LLMs trust AI regulation? Emerging behaviour of game-theoretic LLM agentscited by 11
  23. 2024Responsible Reporting for Frontier AI Developmentcited by 10
  24. 2024Societal Adaptation to Advanced AIcited by 10
  25. 2024AI Alignment through Reinforcement Learning from Human Feedback? Contradictions and Limitationscited by 10
  26. 2024Trust AI Regulation? Discerning users are vital to build trust and effective AI regulationcited by 9
  27. 2024The Necessity of AI Audit Standards Boardscited by 9
  28. 2023International Governance of Civilian AI: A Jurisdictional Certification Approachcited by 8
  29. 2024An Auditing Test To Detect Behavioral Shift in Language Modelscited by 8
  30. 2026Quantifying Trust: Financial Risk Management for Trustworthy AI Agentscited by 8
  31. 2023Hazards from Increasingly Accessible Fine-Tuning of Downloadable Foundation Modelscited by 7
  32. 2024Exploring ChatGPT and its Impact on Societycited by 6
  33. 2026Autonomous Research Agents: A Survey of AI Scientists and the Verification Gapcited by 6
  34. 2023Towards Responsible Governance of Biological Design Toolscited by 5
  35. 2024XLM for Autonomous Driving Systems: A Comprehensive Reviewcited by 5
  36. 2025AI Governance through Marketscited by 5
  37. 2025Real-World Gaps in AI Governance Researchcited by 5
  38. 2026Self-Improvement of Large Language Models: A Technical Overview and Future Outlookcited by 5

50 most cited of 95 All 95 (JSON)