# How do Machine Learning Models Change?

arXiv: https://arxiv.org/abs/2411.09645 · HTML: https://mlsanity.com/paper/2411.09645/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 73 references that are in the map.
- First posted on arXiv 14 Nov 2024.

## Builds on (most cited first, 15 of 15)

- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [An Empirical Study of Pre-Trained Model Reuse in the Hugging Face Deep Learning Model Registry](https://mlsanity.com/paper/2303.02552/) (2023)
- [Exploring the Carbon Footprint of Hugging Face's ML Models: A Repository Mining Study](https://mlsanity.com/paper/2305.11164/) (2023)
- [Analyzing the Evolution and Maintenance of ML Models on Hugging Face](https://mlsanity.com/paper/2311.13380/) (2023)
- [Characterizing Technical Debt and Antipatterns in AI-Based Systems: A Systematic Mapping Study](https://mlsanity.com/paper/2103.09783/) (2021)
- [Learning the Structure of Dynamic Probabilistic Networks](https://mlsanity.com/paper/1301.7374/) (2013)
- [Ecosystem of Large Language Models for Code](https://mlsanity.com/paper/2405.16746/) (2024)
- [PeaTMOSS: A Dataset and Initial Analysis of Pre-Trained Models in Open-Source Software](https://mlsanity.com/paper/2402.00699/) (2024)
- [What do we know about Hugging Face? A systematic literature review and quantitative validation of qualitative claims](https://mlsanity.com/paper/2406.08205/) (2024)
- [Towards a Change Taxonomy for Machine Learning Systems](https://mlsanity.com/paper/2203.11365/) (2022)
- [Towards Semantic Versioning of Open Pre-trained Language Model Releases on Hugging Face](https://mlsanity.com/paper/2409.10472/) (2024)
- [Documenting Ethical Considerations in Open Source AI Models](https://mlsanity.com/paper/2406.18071/) (2024)
- [Watch out for This Commit! A Study of Influential Software Changes](https://mlsanity.com/paper/1606.03266/) (2016)
- [Impact of ML Optimization Tactics on Greener Pre-Trained ML Models](https://mlsanity.com/paper/2409.12878/) (2024)
- [A Longitudinal Cohort Study on the Retainment of Test-Driven Development](https://mlsanity.com/paper/1807.02971/) (2018)

## Built on by (most cited first, 3 of 3)

- [Economies of Open Intelligence: Tracing Power & Participation in the Model Ecosystem](https://mlsanity.com/paper/2512.03073/) (2025)
- [Cataloguing Hugging Face Models to Software Engineering Activities: Automation and Findings](https://mlsanity.com/paper/2506.03013/) (2025)
- [ArtifactLinker: Linking Scientific Artifacts for Automatic State-of-the-Art Discovery](https://mlsanity.com/paper/2605.16902/) (2026)

## Abstract (arXiv)

The proliferation of Machine Learning (ML) models and their open-source implementations has transformed Artificial Intelligence research and applications. Platforms like Hugging Face (HF) enable this evolving ecosystem, yet a large-scale longitudinal study of how these models change is lacking. This study addresses this gap by analyzing over 680,000 commits from 100,000 models and 2,251 releases from 202 of these models on HF using repository mining and longitudinal methods. We apply an extended ML change taxonomy to classify commits and use Bayesian networks to model temporal patterns in commit and release activities. Our findings show that commit activities align with established data science methodologies, such as the Cross-Industry Standard Process for Data Mining (CRISP-DM), emphasizing iterative refinement. Release patterns tend to consolidate significant updates, particularly in model outputs, sharing, and documentation, distinguishing them from granular commits. Furthermore, projects with higher popularity exhibit distinct evolutionary paths, often starting from a more mature baseline with fewer foundational commits in their public history. In contrast, those with intensive collaboration show unique documentation and technical evolution patterns. These insights enhance the understanding of model changes on community platforms and provide valuable guidance for best practices in model maintenance.