# Are Large Language Models a Threat to Digital Public Goods? Evidence from Activity on Stack Overflow

arXiv: https://arxiv.org/abs/2307.07367 · HTML: https://mlsanity.com/paper/2307.07367/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 78 references that are in the map.
- First posted on arXiv 14 Jul 2023.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [OpenAssistant Conversations -- Democratizing Large Language Model Alignment](https://mlsanity.com/paper/2304.07327/) (2023)
- [The Curse of Recursion: Training on Generated Data Makes Models Forget](https://mlsanity.com/paper/2305.17493/) (2023)
- [The Impact of AI on Developer Productivity: Evidence from GitHub Copilot](https://mlsanity.com/paper/2302.06590/) (2023)
- [The False Promise of Imitating Proprietary LLMs](https://mlsanity.com/paper/2305.15717/) (2023)
- [Self-Consuming Generative Models Go MAD](https://mlsanity.com/paper/2307.01850/) (2023)
- [GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models](https://mlsanity.com/paper/2303.10130/) (2023)
- [The Science of Detecting LLM-Generated Texts](https://mlsanity.com/paper/2303.07205/) (2023)
- [Artificial Artificial Artificial Intelligence: Crowd Workers Widely Use Large Language Models for Text Production Tasks](https://mlsanity.com/paper/2306.07899/) (2023)
- [Astronomia ex machina: a history, primer, and outlook on neural networks in astronomy](https://mlsanity.com/paper/2211.03796/) (2022)
- [Why the World Reads Wikipedia: Beyond English Speakers](https://mlsanity.com/paper/1812.00474/) (2018)
- [ChatGPT vs. Google: A Comparative Study of Search Performance and User Experience](https://mlsanity.com/paper/2307.01135/) (2023)
- [On the Value of Wikipedia as a Gateway to the Web](https://mlsanity.com/paper/2102.07385/) (2021)
- [The Dimensions of Data Labor: A Road Map for Researchers, Activists, and Policymakers to Empower Data Producers](https://mlsanity.com/paper/2305.13238/) (2023)
- [Fit or Unfit : Analysis and Prediction of 'Closed Questions' on Stack Overflow](https://mlsanity.com/paper/1307.7291/) (2013)
- [The Unintended Consequences of Censoring Digital Technology -- Evidence from Italy's ChatGPT Ban](https://mlsanity.com/paper/2304.09339/) (2023)
- [Uncovering the Dynamics of Crowdlearning and the Value of Knowledge](https://mlsanity.com/paper/1612.04831/) (2016)
- [Nothing to See Here? A non-inferiority approach to parallel trends](https://mlsanity.com/paper/1805.03273/) (2018)

## Built on by (most cited first, 22 of 22)

- [Sociotechnical Safety Evaluation of Generative AI Systems](https://mlsanity.com/paper/2310.11986/) (2023)
- [How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapse](https://mlsanity.com/paper/2404.05090/) (2024)
- [A Study of LLMs' Preferences for Libraries and Programming Languages](https://mlsanity.com/paper/2503.17181/) (2025)
- [Holistic Safety and Responsibility Evaluations of Advanced AI Models](https://mlsanity.com/paper/2404.14068/) (2024)
- [The Invisible Hand: Unveiling Provider Bias in Large Language Models for Code Generation](https://mlsanity.com/paper/2501.07849/) (2025)
- [Sociotechnical Implications of Generative Artificial Intelligence for Information Access](https://mlsanity.com/paper/2405.11612/) (2024)
- [Smarter Together: Creating Agentic Communities of Practice through Shared Experiential Learning](https://mlsanity.com/paper/2511.08301/) (2025)
- [Impact of AI Search Summaries on Website Traffic: Evidence from Google AI Overviews and Wikipedia](https://mlsanity.com/paper/2602.18455/) (2026)
- [The Heterogeneous Productivity Effects of Generative AI](https://mlsanity.com/paper/2403.01964/) (2024)
- [Balancing Progress and Responsibility: A Synthesis of Sustainability Trade-Offs of AI-Based Systems](https://mlsanity.com/paper/2404.03995/) (2024)
- [Ethical and Scalable Automation: A Governance and Compliance Framework for Business Applications](https://mlsanity.com/paper/2409.16872/) (2024)
- [An Empirical Study of OpenAI API Discussions on Stack Overflow](https://mlsanity.com/paper/2505.04084/) (2025)
- [Library Hallucinations in LLM-Generated Code: A Risk Analysis Grounded in Developer Queries](https://mlsanity.com/paper/2509.22202/) (2025)
- [The Future of Facts: Tracing the Factual Generation-Verification Gap](https://mlsanity.com/paper/2605.27564/) (2026)
- [Security and Privacy Prompts in the Wild: What Users Ask LLMs and How LLMs Respond](https://mlsanity.com/paper/2606.18062/) (2026)
- [The Impact of AI Search on the Online Content Ecosystem: Evidence from Google and Reddit](https://mlsanity.com/paper/2605.16428/) (2026)
- [Exploratory Responsiveness and Adaptive Rigidity under AI-Assisted Optimization](https://mlsanity.com/paper/2606.10086/) (2026)
- [Predictive Assistance and the Temporal Dynamics of Exploratory Compression](https://mlsanity.com/paper/2606.10094/) (2026)
- [Market Design for AI: Beyond the Copyright Binary](https://mlsanity.com/paper/2606.12260/) (2026)
- [The MMM Data Model -- A Normative Specification for Knowledge Interoperability in a Decentralisable Knowledge Commons](https://mlsanity.com/paper/2607.00032/) (2026)
- [You Can Tell Who's Asking: What the Web's Questions Are Made Of, and Where They Come From](https://mlsanity.com/paper/2609.24106/) (2026)
- [The Uneven Decline of Collective Knowledge Production: Evidence from Stack Overflow After Generative AI](https://mlsanity.com/paper/2609.36069/) (2026)

## Abstract (arXiv)

Large language models like ChatGPT efficiently provide users with information about various topics, presenting a potential substitute for searching the web and asking people for help online. But since users interact privately with the model, these models may drastically reduce the amount of publicly available human-generated data and knowledge resources. This substitution can present a significant problem in securing training data for future models. In this work, we investigate how the release of ChatGPT changed human-generated open data on the web by analyzing the activity on Stack Overflow, the leading online Q&A platform for computer programming. We find that relative to its Russian and Chinese counterparts, where access to ChatGPT is limited, and to similar forums for mathematics, where ChatGPT is less capable, activity on Stack Overflow significantly decreased. A difference-in-differences model estimates a 16% decrease in weekly posts on Stack Overflow. This effect increases in magnitude over time, and is larger for posts related to the most widely used programming languages. Posts made after ChatGPT get similar voting scores than before, suggesting that ChatGPT is not merely displacing duplicate or low-quality content. These results suggest that more users are adopting large language models to answer questions and they are better substitutes for Stack Overflow for languages for which they have more training data. Using models like ChatGPT may be more efficient for solving certain programming problems, but its widespread adoption and the resulting shift away from public exchange on the web will limit the open data people and models can learn from in the future.