mlsanityData to 14 Oct 2026

Near to Mid-term Risks and Opportunities of Open-Source Generative AI

arXiv 2404.17047 ↗First posted 25 Apr 2024cs.LGcited by 13

Not on the map. Why?

Abstract (arXiv)

In the next few years, applications of Generative AI are expected to revolutionize a number of different areas, ranging from science & medicine to education. The potential for these seismic changes has triggered a lively debate about potential risks and resulted in calls for tighter regulation, in particular from some of the major tech companies who are leading in AI development. This regulation is likely to put at risk the budding field of open-source Generative AI. We argue for the responsible open sourcing of generative AI models in the near and medium term. To set the stage, we first introduce an AI openness taxonomy system and apply it to 40 current large language models. We then outline differential benefits and risks of open versus closed source AI and present potential risk mitigation, ranging from best practices to calls for technical and scientific contributions. We hope that this report will add a much needed missing voice to the current public discourse on near to mid-term AI safety and other societal impact.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 64 of its 144 references that are in the map. First posted on arXiv 25 Apr 2024.

Builds on

64 of 144 references.

References · 64

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2017Proximal Policy Optimization Algorithmscited by 15,215
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  7. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  8. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  9. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  10. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  11. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
  12. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
38 more
  1. 2018Datasheets for Datasetscited by 1,386
  2. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,254
  3. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  4. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,202
  5. 2023PaLM 2 Technical Reportcited by 1,162
  6. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
  7. 2023LIMA: Less Is More for Alignmentcited by 1,100
  8. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 798
  9. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 559
  10. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 434
  11. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Ciphercited by 423
  12. 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 387
  13. 2022A Systematic Evaluation of Large Language Models of Codecited by 363
  14. 2023NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmarkcited by 348
  15. 2023ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledgecited by 325
  16. 2023The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variantscited by 311
  17. 2023Low-Resource Languages Jailbreak GPT-4cited by 309
  18. 2020Large image datasets: A pyrrhic win for computer vision?cited by 237
  19. 2023An Overview of Catastrophic AI Riskscited by 222
  20. 2023Language Model Tokenizers Introduce Unfairness Between Languagescited by 214
  21. 2023Evaluating Large Language Models: A Comprehensive Surveycited by 202
  22. 2023Don't Make Your LLM an Evaluation Benchmark Cheatercited by 194
  23. 2023Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Modelscited by 192
  24. 2023Model evaluation for extreme riskscited by 177
  25. 2023Sociotechnical Safety Evaluation of Generative AI Systemscited by 157
  26. 2022Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reductioncited by 138
  27. 2018Explaining Explanations in AIcited by 137
  28. 2023Auditing large language models: a three-layered approachcited by 137
  29. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 126
  30. 2023Mechanistically analyzing the effects of fine-tuning on procedurally defined taskscited by 115
  31. 2023The Impact of Large Language Models on Scientific Discovery: a Preliminary Study using GPT-4cited by 112
  32. 2023Regulating ChatGPT and other Large Generative AI Modelscited by 104
  33. 2023LLM360: Towards Fully Transparent Open-Source LLMscited by 104
  34. 2023Evaluating the Social Impact of Generative AI Systems in Systems and Societycited by 92
  35. 2023Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedbackcited by 87
  36. 2023ChatGPT is not all you need. A State of the Art Review of large Generative AI modelscited by 86
  37. 2023DICES Dataset: Diversity in Conversational AI Evaluation for Safetycited by 86
  38. 2023A Systematic Review of Green AIcited by 77

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 7; 2026: 1.

13 papers.

Sort:

By area:

  1. 2024Language Models Resist Alignment: Evidence From Data Compressioncited by 53
  2. 2024AlignGuard: Scalable Safety Alignment for Text-to-Image Generationcited by 20
  3. 2025An Economy of AI Agentscited by 18
  4. 2024Risks and Opportunities of Open-Source Generative AIcited by 13
  5. 2026When Safety Collides: Resolving Multi-Category Harmful Conflicts in Text-to-Image Diffusion via Adaptive Safety Guidancecited by 11
  6. 2025Position: Collaborative Agentic AI Needs Interoperability Across Ecosystemscited by 8
  7. 2025MusGO: A Community-Driven Framework For Assessing Openness in Music-Generative AIcited by 7
  8. 2024Comparative Global AI Regulation: Policy Perspectives from the EU, China, and the UScited by 5
  9. 2025Toward a Public and Secure Generative AI: A Comparative Analysis of Open and Closed LLMscited by 3
  10. 2024Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systemscited by 2
  11. 2025From Firewalls to Frontiers: AI Red-Teaming is a Domain-Specific Evolution of Cyber Red-Teamingcited by 2
  12. 2025Beyond the Cloud: Assessing the Benefits and Drawbacks of Local LLM Deployment for Translatorscited by 1
1 more
  1. 2025Tahakom LLM Guidelines and Recipes: From Pre-training Data to an Arabic LLMcited by 0