# An Empirical Analysis of Machine Learning Model and Dataset Documentation, Supply Chain, and Licensing Challenges on Hugging Face arXiv: https://arxiv.org/abs/2502.04484 ยท HTML: https://mlsanity.com/paper/2502.04484/ ## Key facts - Cited by 6 papers in the map (data through 14 Oct 2026). - Builds on 34 of its 118 references that are in the map. - First posted on arXiv 6 Feb 2025. ## Builds on (most cited first, 30 of 34) - [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015) - [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018) - [Carbon Emissions and Large Neural Network Training](https://mlsanity.com/paper/2104.10350/) (2021) - [Learning to (Learn at Test Time): RNNs with Expressive Hidden States](https://mlsanity.com/paper/2407.04620/) (2024) - [Weight Poisoning Attacks on Pre-trained Models](https://mlsanity.com/paper/2004.06660/) (2020) - [Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunities](https://mlsanity.com/paper/2408.07666/) (2024) - [Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses](https://mlsanity.com/paper/2012.10544/) (2020) - [The Data Provenance Initiative: A Large Scale Audit of Dataset Licensing & Attribution in AI](https://mlsanity.com/paper/2310.16787/) (2023) - [An Empirical Study of Pre-Trained Model Reuse in the Hugging Face Deep Learning Model Registry](https://mlsanity.com/paper/2303.02552/) (2023) - [Talkin' 'Bout AI Generation: Copyright and the Generative-AI Supply Chain](https://mlsanity.com/paper/2309.08133/) (2023) - [Navigating Dataset Documentations in AI: A Large-Scale Analysis of Dataset Cards on Hugging Face](https://mlsanity.com/paper/2401.13822/) (2024) - [Exploring the Carbon Footprint of Hugging Face's ML Models: A Repository Mining Study](https://mlsanity.com/paper/2305.11164/) (2023) - [The Model Openness Framework: Promoting Completeness and Openness for Reproducibility, Transparency, and Usability in Artificial Intelligence](https://mlsanity.com/paper/2403.13784/) (2024) - [Analyzing the Evolution and Maintenance of ML Models on Hugging Face](https://mlsanity.com/paper/2311.13380/) (2023) - [EvilModel: Hiding Malware Inside of Neural Network Models](https://mlsanity.com/paper/2107.08590/) (2021) - [Aspirations and Practice of Model Documentation: Moving the Needle with Nudging and Traceability](https://mlsanity.com/paper/2204.06425/) (2022) - [Towards Traceability in Data Ecosystems using a Bill of Materials Model](https://mlsanity.com/paper/1904.04253/) (2019) - [Deep Learning Model Reuse in the HuggingFace Community: Challenges, Benefit and Trends](https://mlsanity.com/paper/2401.13177/) (2024) - [Challenges and Practices of Deep Learning Model Reengineering: A Case Study on Computer Vision](https://mlsanity.com/paper/2303.07476/) (2023) - [Reusing Deep Learning Models: Challenges and Directions in Software Engineering](https://mlsanity.com/paper/2404.16688/) (2024) - [What do we know about Hugging Face? A systematic literature review and quantitative validation of qualitative claims](https://mlsanity.com/paper/2406.08205/) (2024) - [Trust in Software Supply Chains: Blockchain-Enabled SBOM and the AIBOM Future](https://mlsanity.com/paper/2307.02088/) (2023) - [PTMTorrent: A Dataset for Mining Open-source Pre-trained Model Packages](https://mlsanity.com/paper/2303.08934/) (2023) - [BOMs Away! Inside the Minds of Stakeholders: A Comprehensive Study of Bills of Materials for Software Systems](https://mlsanity.com/paper/2309.12206/) (2023) - [Lessons Learned from Mining the Hugging Face Repository](https://mlsanity.com/paper/2402.07323/) (2024) - [A Taxonomy of Foundation Model based Systems through the Lens of Software Architecture](https://mlsanity.com/paper/2305.05352/) (2023) - [The Use of Synthetic Data to Train AI Models: Opportunities and Risks for Sustainable Development](https://mlsanity.com/paper/2309.00652/) (2023) - [The State of Documentation Practices of Third-party Machine Learning Models and Datasets](https://mlsanity.com/paper/2312.15058/) (2023) - [SoK: Analysis of Software Supply Chain Security by Establishing Secure Design Properties](https://mlsanity.com/paper/2406.10109/) (2024) - [On the Way to SBOMs: Investigating Design Issues and Solutions in Practice](https://mlsanity.com/paper/2304.13261/) (2023) ## Built on by (most cited first, 6 of 6) - [TensorDex: A Compact, Tensor-Centric Storage System for Modern AI Models](https://mlsanity.com/paper/2604.17104/) (2026) - [A governance horizon for ethical-use constraints in open-weight AI models](https://mlsanity.com/paper/2605.24383/) (2026) - [Stemma: Induced Decision Regions Reveal LLM Provenance](https://mlsanity.com/paper/2607.25880/) (2026) - [A Security-Oriented Lifecycle Model for Large Language Model Systems](https://mlsanity.com/paper/2608.03626/) (2026) - [After the Party: Growth, Governance, and Security Scanning in the OpenClaw Agent Skill Ecosystem](https://mlsanity.com/paper/2609.17274/) (2026) - [Witness Overlap: Directional Provenance Inside Open-Weight Model Families](https://mlsanity.com/paper/2609.31784/) (2026) ## Abstract (arXiv) The last decade has seen widespread adoption of Machine Learning (ML) components in software systems. This has occurred in nearly every domain, from natural language processing to computer vision. These ML components range from relatively simple neural networks to complex and resource-intensive large language models. However, despite this widespread adoption, little is known about the supply chain relationships that produce these models, which can have implications for compliance and security. In this work, we conducted an extensive analysis of 760,460 models and 175,000 datasets extracted from the popular model-sharing site Hugging Face. First, we evaluate the current state of documentation in the Hugging Face supply chain, report real-world examples of shortcomings, and offer actionable suggestions for improvement. Next, we analyze the underlying structure of the existing supply chain. Finally, we explore the current licensing landscape against what was reported in previous work and discuss the unique challenges posed in this domain. Our results motivate multiple research avenues, including the need for better license management for ML models/datasets, better support for model documentation, and automated inconsistency checking and validation. We make our research infrastructure and dataset available to facilitate future research.