How do Machine Learning Models Change?
Not on the map. Why?
Abstract (arXiv)
The proliferation of Machine Learning (ML) models and their open-source implementations has transformed Artificial Intelligence research and applications. Platforms like Hugging Face (HF) enable this evolving ecosystem, yet a large-scale longitudinal study of how these models change is lacking. This study addresses this gap by analyzing over 680,000 commits from 100,000 models and 2,251 releases from 202 of these models on HF using repository mining and longitudinal methods. We apply an extended ML change taxonomy to classify commits and use Bayesian networks to model temporal patterns in commit and release activities. Our findings show that commit activities align with established data science methodologies, such as the Cross-Industry Standard Process for Data Mining (CRISP-DM), emphasizing iterative refinement. Release patterns tend to consolidate significant updates, particularly in model outputs, sharing, and documentation, distinguishing them from granular commits. Furthermore, projects with higher popularity exhibit distinct evolutionary paths, often starting from a more mature baseline with fewer foundational commits in their public history. In contrast, those with intensive collaboration show unique documentation and technical evolution patterns. These insights enhance the understanding of model changes on community platforms and provide valuable guidance for best practices in model maintenance.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 15 of its 73 references that are in the map. First posted on arXiv 14 Nov 2024.
Builds on
15 of 73 references.
References · 15
- 2020Measuring Massive Multitask Language Understandingcited by 7,963
- 2023An Empirical Study of Pre-Trained Model Reuse in the Hugging Face Deep Learning Model Registrycited by 47
- 2023Exploring the Carbon Footprint of Hugging Face's ML Models: A Repository Mining Studycited by 23
- 2023Analyzing the Evolution and Maintenance of ML Models on Hugging Facecited by 22
- 2021Characterizing Technical Debt and Antipatterns in AI-Based Systems: A Systematic Mapping Studycited by 21
- 2013Learning the Structure of Dynamic Probabilistic Networkscited by 12
- 2024Ecosystem of Large Language Models for Codecited by 10
- 2024PeaTMOSS: A Dataset and Initial Analysis of Pre-Trained Models in Open-Source Softwarecited by 9
- 2024What do we know about Hugging Face? A systematic literature review and quantitative validation of qualitative claimscited by 9
- 2022Towards a Change Taxonomy for Machine Learning Systemscited by 6
- 2024Towards Semantic Versioning of Open Pre-trained Language Model Releases on Hugging Facecited by 6
- 2024Documenting Ethical Considerations in Open Source AI Modelscited by 4
Built on by
Citing papers per year: 2025: 2; 2026: 1.
3 papers.
- 2025Economies of Open Intelligence: Tracing Power & Participation in the Model Ecosystemcited by 4
- 2025Cataloguing Hugging Face Models to Software Engineering Activities: Automation and Findingscited by 1
- 2026ArtifactLinker: Linking Scientific Artifacts for Automatic State-of-the-Art Discoverycited by 0
- 2026ArtifactLinker: Linking Scientific Artifacts for Automatic State-of-the-Art Discovery16 May 2026
- 2025Economies of Open Intelligence: Tracing Power & Participation in the Model Ecosystem27 Nov 2025
- 2025Cataloguing Hugging Face Models to Software Engineering Activities: Automation and Findings3 Jun 2025