Acceptable Use Policies for Foundation Models
Not on the map. Why?
Abstract (arXiv)
As foundation models have accumulated hundreds of millions of users, developers have begun to take steps to prevent harmful types of uses. One salient intervention that foundation model developers adopt is acceptable use policies: legally binding policies that prohibit users from using a model for specific purposes. This paper identifies acceptable use policies from 30 foundation model developers, analyzes the use restrictions they contain, and argues that acceptable use policies are an important lens for understanding the regulation of foundation models. Taken together, developers' acceptable use policies include 127 distinct use restrictions; the wide variety in the number and type of use restrictions may create fragmentation across the AI supply chain. Developers also employ acceptable use policies to prevent competitors or specific industries from making use of their models. Developers alone decide what constitutes acceptable use, and rarely provide transparency about how they enforce their policies. In practice, acceptable use policies are difficult to enforce, and scrupulous enforcement can act as a barrier to researcher access and limit beneficial uses of foundation models. Nevertheless, acceptable use policies for foundation models are an early example of self-regulation that have a significant impact on the market for foundation models and the overall AI ecosystem.
Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 65 of its 182 references that are in the map. First posted on arXiv 29 Aug 2024.
Builds on
65 of 182 references.
References · 65
- 2017Deep reinforcement learning from human preferencescited by 3,846
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
- 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
- 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
- 2018Model Cards for Model Reportingcited by 1,068
- 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 703
- 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 657
- 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 598
- 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 505
38 more
- 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Modelscited by 457
- 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 434
- 2024Mixture-of-Agents Enhances Large Language Model Capabilitiescited by 420
- 2023Shadow Alignment: The Ease of Subverting Safely-Aligned Language Modelscited by 284
- 2023Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulationcited by 202
- 2023Removing RLHF Protections in GPT-4 via Fine-Tuningcited by 165
- 2023Generative Language Models and Automated Influence Operations: Emerging Threats and Potential Mitigationscited by 157
- 2023Sociotechnical Safety Evaluation of Generative AI Systemscited by 154
- 2024Large Legal Fictions: Profiling Legal Hallucinations in Large Language Modelscited by 144
- 2022Power to the People? Opportunities and Challenges for Participatory AIcited by 137
- 2022Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reductioncited by 137
- 2024Collective Constitutional AI: Aligning a Language Model with Public Inputcited by 120
- 2023Evaluating and Mitigating Discrimination in Language Model Decisionscited by 117
- 2023Foundation Models and Fair Usecited by 112
- 2022Fusing finetuned models for better pretrainingcited by 110
- 2024The Art of Saying No: Contextual Noncompliance in Language Modelscited by 100
- 2023The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practicecited by 86
- 2024AIR-Bench 2024: A Safety Benchmark Based on Risk Categories from Regulations and Policiescited by 69
- 2024On the Societal Impact of Open Foundation Modelscited by 64
- 2023The Gradient of Generative AI Release: Methods and Considerationscited by 63
- 2023Black Box Adversarial Prompting for Foundation Modelscited by 62
- 2022Accountability in an Algorithmic Society: Relationality, Responsibility, and Robustness in Machine Learningcited by 47
- 2020Behavioral Use Licensing for Responsible AIcited by 46
- 2023The History and Risks of Reinforcement Learning and Human Feedbackcited by 42
- 2024AI Risk Categorization Decoded (AIR 2024): From Government Regulations to Corporate Policiescited by 34
- 2024Participation in the age of foundation modelscited by 32
- 2023Ecosystem Graphs: The Social Footprint of Foundation Modelscited by 31
- 2023Democratising AI: Multiple Meanings, Goals, and Methodscited by 29
- 2024Risks from Language Models for Automated Mental Healthcare: Ethics and Structure for Implementationcited by 27
- 2024Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Datacited by 20
- 2022Reward Reports for Reinforcement Learningcited by 18
- 2023The Ethical Implications of Generative Audio Models: A Systematic Literature Reviewcited by 18
- 2024The Social Impact of Generative AI: An Analysis on ChatGPTcited by 18
- 2024AI-Powered Autonomous Weapons Risk Geopolitical Instability and Threaten AI Researchcited by 13
- 2024STAR: SocioTechnical Approach to Red Teaming Language Modelscited by 13
- 2023Where's the Liability in Harmful AI Speech?cited by 12
- 2024Risks and Opportunities of Open-Source Generative AIcited by 11
- 2024What's documented in AI? Systematic Analysis of 32K AI Model Cardscited by 10
50 most cited of 65 All 65 (JSON)
Built on by
Citing papers per year: 2024: 1; 2025: 6; 2026: 5.
12 papers.
- 2025Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providerscited by 67
- 2026Characterizing Delusional Spirals through Human-LLM Chat Logscited by 22
- 2024Machine Unlearning Doesn't Do What You Think: Lessons for Generative AI Policy and Researchcited by 14
- 2025In-House Evaluation Is Not Enough: Towards Robust Third-Party Flaw Disclosure for General-Purpose AIcited by 7
- 2025Breaking Down Bias: On The Limits of Generalizable Pruning Strategiescited by 2
- 2026What if AI systems weren't chatbots?cited by 2
- 2025Policy-Grounded Safety Evaluation of 20 Large Language Modelscited by 1
- 2025From Symptoms to Systems: An Expert-Guided Approach to Understanding Risks of Generative AI for Eating Disorderscited by 1
- 2025From Rogue to Safe AI: The Role of Explicit Refusals in Aligning LLMs with International Humanitarian Lawcited by 0
- 2026Cripping AI: Reimagining AI Through Lived Disability Experiencescited by 0
- 2026FLARE-AI: Flaw Reporting for AIcited by 0
- 2026Position: Current Model Cards Are Insufficient for Downstream Governance of Open-Weight Foundation Modelscited by 0
- 2026FLARE-AI: Flaw Reporting for AI30 Jun 2026
- 2026Position: Current Model Cards Are Insufficient for Downstream Governance of Open-Weight Foundation Models5 Jun 2026
- 2026What if AI systems weren't chatbots?8 May 2026
- 2026Cripping AI: Reimagining AI Through Lived Disability Experiences3 May 2026
- 2026Characterizing Delusional Spirals through Human-LLM Chat Logs17 Mar 2026
- 2025From Symptoms to Systems: An Expert-Guided Approach to Understanding Risks of Generative AI for Eating Disorders4 Dec 2025
- 2025Policy-Grounded Safety Evaluation of 20 Large Language Models19 Jul 2025
- 2025From Rogue to Safe AI: The Role of Explicit Refusals in Aligning LLMs with International Humanitarian Law5 Jun 2025
- 2025Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providers25 Apr 2025
- 2025In-House Evaluation Is Not Enough: Towards Robust Third-Party Flaw Disclosure for General-Purpose AI21 Mar 2025
- 2025Breaking Down Bias: On The Limits of Generalizable Pruning Strategies11 Feb 2025
- 2024Machine Unlearning Doesn't Do What You Think: Lessons for Generative AI Policy and Research9 Dec 2024