# Moderating Model Marketplaces: Platform Governance Puzzles for AI Intermediaries

arXiv: https://arxiv.org/abs/2311.12573 · HTML: https://mlsanity.com/paper/2311.12573/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 119 references that are in the map.
- First posted on arXiv 21 Nov 2023.

## Builds on (most cited first, 20 of 20)

- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [Quantifying Memorization Across Neural Language Models](https://mlsanity.com/paper/2202.07646/) (2022)
- [Who's Harry Potter? Approximate Unlearning in LLMs](https://mlsanity.com/paper/2310.02238/) (2023)
- [Deduplicating Training Data Mitigates Privacy Risks in Language Models](https://mlsanity.com/paper/2202.06539/) (2022)
- [Stable Bias: Analyzing Societal Representations in Diffusion Models](https://mlsanity.com/paper/2303.11408/) (2023)
- [Challenges in Detoxifying Language Models](https://mlsanity.com/paper/2109.07445/) (2021)
- [A Review on Language Models as Knowledge Bases](https://mlsanity.com/paper/2204.06031/) (2022)
- [Machine Learning Operations (MLOps): Overview, Definition, and Architecture](https://mlsanity.com/paper/2205.02302/) (2022)
- [Regulating ChatGPT and other Large Generative AI Models](https://mlsanity.com/paper/2302.02337/) (2023)
- [Selective Differential Privacy for Language Modeling](https://mlsanity.com/paper/2108.12944/) (2021)
- [Algorithms that Remember: Model Inversion Attacks and Data Protection Law](https://mlsanity.com/paper/1807.04644/) (2018)
- [LLM Censorship: A Machine Learning Challenge or a Computer Security Problem?](https://mlsanity.com/paper/2307.10719/) (2023)
- [Behavioral Use Licensing for Responsible AI](https://mlsanity.com/paper/2011.03116/) (2020)
- [Understanding accountability in algorithmic supply chains](https://mlsanity.com/paper/2304.14749/) (2023)
- [Raiders of the Lost Kek: 3.5 Years of Augmented 4chan Posts from the Politically Incorrect Board](https://mlsanity.com/paper/2001.07487/) (2020)
- [Safety and Fairness for Content Moderation in Generative Models](https://mlsanity.com/paper/2306.06135/) (2023)
- [Stronger Together: on the Articulation of Ethical Charters, Legal Tools, and Technical Documentation in ML](https://mlsanity.com/paper/2305.18615/) (2023)
- [Understanding the Complexity of Detecting Political Ads](https://mlsanity.com/paper/2103.00822/) (2021)
- [Do You See What I See? Capabilities and Limits of Automated Multimedia Content Analysis](https://mlsanity.com/paper/2201.11105/) (2021)

## Built on by (most cited first, 9 of 9)

- [The AI Community Building the Future? A Quantitative Analysis of Development Activity on Hugging Face Hub](https://mlsanity.com/paper/2405.13058/) (2024)
- [Acceptable Use Policies for Foundation Models](https://mlsanity.com/paper/2409.09041/) (2024)
- [Law and the Emerging Political Economy of Algorithmic Audits](https://mlsanity.com/paper/2406.11855/) (2024)
- [Permissive-Washing in the Open AI Supply Chain: A Large-Scale Audit of License Integrity](https://mlsanity.com/paper/2602.08816/) (2026)
- [Regulating Artificial Intimacy: From Locks and Blocks to Relational Accountability](https://mlsanity.com/paper/2604.18893/) (2026)
- [A Cartography of Open Collaboration in Open Source AI: Mapping Practices, Motivations, and Governance in 14 Open Large Language Model Projects](https://mlsanity.com/paper/2509.25397/) (2025)
- [PoLO: Proof-of-Learning and Proof-of-Ownership at Once with Chained Watermarking](https://mlsanity.com/paper/2505.12296/) (2025)
- [Position: EU AI Act's Research Exemptions Can Break the Publication Norms of Major AI Conferences](https://mlsanity.com/paper/2506.03218/) (2025)
- [A governance horizon for ethical-use constraints in open-weight AI models](https://mlsanity.com/paper/2605.24383/) (2026)

## Abstract (arXiv)

The AI development community is increasingly making use of hosting intermediaries such as Hugging Face provide easy access to user-uploaded models and training data. These model marketplaces lower technical deployment barriers for hundreds of thousands of users, yet can be used in numerous potentially harmful and illegal ways. In this article, we explain ways in which AI systems, which can both `contain' content and be open-ended tools, present one of the trickiest platform governance challenges seen to date. We provide case studies of several incidents across three illustrative platforms -- Hugging Face, GitHub and Civitai -- to examine how model marketplaces moderate models. Building on this analysis, we outline important (and yet nevertheless limited) practices that industry has been developing to respond to moderation demands: licensing, access and use restrictions, automated content moderation, and open policy development. While the policy challenge at hand is a considerable one, we conclude with some ideas as to how platforms could better mobilize resources to act as a careful, fair, and proportionate regulatory access point.