# HuggingGraph: Understanding the Supply Chain of LLM Ecosystem

arXiv: https://arxiv.org/abs/2507.14240 · HTML: https://mlsanity.com/paper/2507.14240/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 73 references that are in the map.
- First posted on arXiv 17 Jul 2025.

## Builds on (most cited first, 27 of 27)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Pre-Trained Models: Past, Present and Future](https://mlsanity.com/paper/2106.07139/) (2021)
- [State of What Art? A Call for Multi-Prompt LLM Evaluation](https://mlsanity.com/paper/2401.00595/) (2023)
- [A Survey on Deep Learning for Named Entity Recognition](https://mlsanity.com/paper/1812.09449/) (2018)
- [Evolutionary Optimization of Model Merging Recipes](https://mlsanity.com/paper/2403.13187/) (2024)
- [Large Language Models as General Pattern Machines](https://mlsanity.com/paper/2307.04721/) (2023)
- [Will we run out of data? Limits of LLM scaling based on human-generated data](https://mlsanity.com/paper/2211.04325/) (2022)
- [Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoders](https://mlsanity.com/paper/2410.20526/) (2024)
- [A Survey of Resource-efficient LLM and Multimodal Foundation Models](https://mlsanity.com/paper/2401.08092/) (2024)
- [SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety](https://mlsanity.com/paper/2404.05399/) (2024)
- [SlimPajama-DC: Understanding Data Combinations for LLM Training](https://mlsanity.com/paper/2309.10818/) (2023)
- [Kaggle forecasting competitions: An overlooked learning opportunity](https://mlsanity.com/paper/2009.07701/) (2020)
- [CodeLMSec Benchmark: Systematically Evaluating and Finding Security Vulnerabilities in Black-Box Code Language Models](https://mlsanity.com/paper/2302.04012/) (2023)
- [An Empirical Study of Pre-Trained Model Reuse in the Hugging Face Deep Learning Model Registry](https://mlsanity.com/paper/2303.02552/) (2023)
- [Towards Few-Shot Adaptation of Foundation Models via Multitask Finetuning](https://mlsanity.com/paper/2402.15017/) (2024)
- [Large Language Model Supply Chain: A Research Agenda](https://mlsanity.com/paper/2404.12736/) (2024)
- [Increasing the LLM Accuracy for Question Answering: Ontologies to the Rescue!](https://mlsanity.com/paper/2405.11706/) (2024)
- [The AI Community Building the Future? A Quantitative Analysis of Development Activity on Hugging Face Hub](https://mlsanity.com/paper/2405.13058/) (2024)
- [An Empirical Study on Using Large Language Models to Analyze Software Supply Chain Security Failures](https://mlsanity.com/paper/2308.04898/) (2023)
- [Quadapter: Adapter for GPT-2 Quantization](https://mlsanity.com/paper/2211.16912/) (2022)
- [PathoTune: Adapting Visual Foundation Model to Pathological Specialists](https://mlsanity.com/paper/2403.16497/) (2024)
- [Assessing the Zero-Shot Capabilities of LLMs for Action Evaluation in RL](https://mlsanity.com/paper/2409.12798/) (2024)
- [LLM-based Interaction for Content Generation: A Case Study on the Perception of Employees in an IT department](https://mlsanity.com/paper/2304.09064/) (2023)
- [A Comprehensive Evaluation of Parameter-Efficient Fine-Tuning on Software Engineering Tasks](https://mlsanity.com/paper/2312.15614/) (2023)
- [Challenges of Using Pre-trained Models: the Practitioners' Perspective](https://mlsanity.com/paper/2404.14710/) (2024)
- [Bingo: Radix-based Bias Factorization for Random Walk on Dynamic Graphs](https://mlsanity.com/paper/2504.10233/) (2025)

## Built on by (most cited first, 8 of 8)

- [LLM DNA: Tracing Model Evolution via Functional Representations](https://mlsanity.com/paper/2509.24496/) (2025)
- [Anatomy of a Machine Learning Ecosystem: 2 Million Models on Hugging Face](https://mlsanity.com/paper/2508.06811/) (2025)
- [Learning a Zeroth-Order Optimizer for Fine-Tuning LLMs](https://mlsanity.com/paper/2510.00419/) (2025)
- [Test Before You Deploy: Governing Updates in the LLM Supply Chain](https://mlsanity.com/paper/2604.27789/) (2026)
- [Which Models Are Our Models Built On? Auditing Invisible Dependencies in Modern LLMs](https://mlsanity.com/paper/2606.12385/) (2026)
- [MetaGAI: A Large-Scale and High-Quality Benchmark for Generative AI Model and Data Card Generation](https://mlsanity.com/paper/2604.23539/) (2026)
- [ArtifactLinker: Linking Scientific Artifacts for Automatic State-of-the-Art Discovery](https://mlsanity.com/paper/2605.16902/) (2026)
- [Witness Overlap: Directional Provenance Inside Open-Weight Model Families](https://mlsanity.com/paper/2609.31784/) (2026)

## Abstract (arXiv)

Large language models (LLMs) leverage deep learning architectures to process and predict sequences of words, enabling them to perform a wide range of natural language processing tasks, such as translation, summarization, question answering, and content generation. As existing LLMs are often built from base models or other pre-trained models and use external datasets, they can inevitably inherit vulnerabilities, biases, or malicious components that exist in previous models or datasets. Therefore, it is critical to understand these components' origin and development process to detect potential risks, improve model fairness, and ensure compliance with regulatory frameworks. Motivated by that, this project aims to study such relationships between models and datasets, which are the central parts of the LLM supply chain. First, we design a methodology to systematically collect LLMs' supply chain information. Then, we design a new graph to model the relationships between models and datasets, which is a directed heterogeneous graph, having 402,654 nodes and 462,524 edges. Lastly, we perform different types of analysis and make multiple interesting findings.