mlsanityData to 7 Oct 2026

PeaTMOSS: A Dataset and Initial Analysis of Pre-Trained Models in Open-Source Software

arXiv 2402.00699 ↗First posted 1 Feb 2024cs.SE cs.AI cs.DBcited by 9

Not on the map. Why?

Abstract (arXiv)

The development and training of deep learning models have become increasingly costly and complex. Consequently, software engineers are adopting pre-trained models (PTMs) for their downstream applications. The dynamics of the PTM supply chain remain largely unexplored, signaling a clear need for structured datasets that document not only the metadata but also the subsequent applications of these models. Without such data, the MSR community cannot comprehensively understand the impact of PTM adoption and reuse. This paper presents the PeaTMOSS dataset, which comprises metadata for 281,638 PTMs and detailed snapshots for all PTMs with over 50 monthly downloads (14,296 PTMs), along with 28,575 open-source software repositories from GitHub that utilize these models. Additionally, the dataset includes 44,337 mappings from 15,129 downstream GitHub repositories to the 2,530 PTMs they use. To enhance the dataset's comprehensiveness, we developed prompts for a large language model to automatically extract model metadata, including the model's training datasets, parameters, and evaluation metrics. Our analysis of this dataset provides the first summary statistics for the PTM supply chain, showing the trend of PTM development and common shortcomings of PTM package documentation. Our example application reveals inconsistencies in software licenses across PTMs and their dependent projects. PeaTMOSS lays the foundation for future research, offering rich opportunities to investigate the PTM supply chain. We outline mining opportunities on PTMs, their downstream usage, and cross-cutting questions.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 36 of its 102 references that are in the map. First posted on arXiv 1 Feb 2024.

Builds on

36 of 102 references.

References · 36

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  3. 2019A Comprehensive Survey on Transfer Learningcited by 1,154
  4. 2020Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditingcited by 380
  5. 2021Pre-Trained Models: Past, Present and Futurecited by 319
  6. 2017Data Augmentation Generative Adversarial Networkscited by 278
  7. 2021AMMUS : A Survey of Transformer-based Pretrained Models in Natural Language Processingcited by 131
  8. 2021CURE: Code-Aware Neural Machine Translation for Automatic Program Repaircited by 85
  9. 2020Machine Learning in Python: Main developments and technology trends in data science, machine learning, and artificial intelligencecited by 81
  10. 2022Cramming: Training a Language Model on a Single GPU in One Daycited by 81
  11. 2018On Testing Machine Learning Programscited by 48
  12. 2020Behavioral Use Licensing for Responsible AIcited by 46
24 more
  1. 2022A Systematic Survey of Chemical Pre-trained Modelscited by 34
  2. 2021ModelDiff: Testing-Based DNN Similarity Comparison for Model Reuse Detectioncited by 33
  3. 2016Understanding the Factors that Impact the Popularity of GitHub Repositoriescited by 27
  4. 2023Exploring the Carbon Footprint of Hugging Face's ML Models: A Repository Mining Studycited by 23
  5. 2023Analyzing the Evolution and Maintenance of ML Models on Hugging Facecited by 22
  6. 2021The Prevalence of Code Smells in Machine Learning projectscited by 16
  7. 2021EvilModel 2.0: Bringing Neural Network Models into Malware Attackscited by 13
  8. 2019Machine Learning Software Engineering in Practice: An Industrial Case Studycited by 12
  9. 2024Deep Learning Model Reuse in the HuggingFace Community: Challenges, Benefit and Trendscited by 12
  10. 2023Challenges and Practices of Deep Learning Model Reengineering: A Case Study on Computer Visioncited by 11
  11. 2018SOTorrent: Reconstructing and Analyzing the Evolution of Stack Overflow Postscited by 9
  12. 2022Scalpel: The Python Static Analysis Frameworkcited by 9
  13. 2019Understanding Flaky Tests: The Developer's Perspectivecited by 8
  14. 2023What Do Users Ask in Open-Source AI Repositories? An Empirical Study of GitHub Issuescited by 8
  15. 2022Code Smells for Machine Learning Applicationscited by 7
  16. 2023PTMTorrent: A Dataset for Mining Open-source Pre-trained Model Packagescited by 7
  17. 2018Usage and Attribution of Stack Overflow Code Snippets in GitHub Projectscited by 6
  18. 2020What Makes a Popular Academic AI Repository?cited by 6
  19. 2022Towards a Change Taxonomy for Machine Learning Systemscited by 6
  20. 2020A Study of Potential Code Borrowing and License Violations in Java Projects on GitHubcited by 5
  21. 2023Discrepancies among Pre-trained Deep Neural Networks: A New Threat to Model Zoo Reliabilitycited by 4
  22. 2023Prevalence of Code Smells in Reinforcement Learning Projectscited by 3
  23. 2023Understanding and Remediating Open-Source License Incompatibilities in the PyPI Ecosystemcited by 2
  24. 2023A Metadata-Based Ecosystem to Improve the FAIRness of Research Softwarecited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 1; 2026: 5.

9 papers.

  1. 2024What do we know about Hugging Face? A systematic literature review and quantitative validation of qualitative claimscited by 8
  2. 2024A Large-Scale Study of Model Integration in ML-Enabled Software Systemscited by 6
  3. 2024How do Machine Learning Models Change?cited by 3
  4. 2026Permissive-Washing in the Open AI Supply Chain: A Large-Scale Audit of License Integritycited by 3
  5. 2026An Empirical Investigation of Pre-Trained Deep Learning Model Reuse in the Scientific Processcited by 1
  6. 2026From Registry to Repository: How AI Agent Skills Are Written, Adapted, and Maintainedcited by 1
  7. 2025AgentHub: A Registry for Discoverable, Verifiable, and Reproducible AI Agentscited by 0
  8. 2026A governance horizon for ethical-use constraints in open-weight AI modelscited by 0
  9. 2026The evolution of sex for artificial intelligence: a population-genetic framework for multigenerational model populationscited by 0