# Large Language Model Supply Chain: Open Problems From the Security Perspective

arXiv: https://arxiv.org/abs/2411.01604 · HTML: https://mlsanity.com/paper/2411.01604/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 27 references that are in the map.
- First posted on arXiv 3 Nov 2024.

## Builds on (most cited first, 17 of 17)

- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation](https://mlsanity.com/paper/2305.01210/) (2023)
- [LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day](https://mlsanity.com/paper/2306.00890/) (2023)
- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [LESS: Selecting Influential Data for Targeted Instruction Tuning](https://mlsanity.com/paper/2402.04333/) (2024)
- [DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model](https://mlsanity.com/paper/2310.01412/) (2023)
- [Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks](https://mlsanity.com/paper/2103.14749/) (2021)
- [Want To Reduce Labeling Cost? GPT-3 Can Help](https://mlsanity.com/paper/2108.13487/) (2021)
- [How Robust is Google's Bard to Adversarial Image Attacks?](https://mlsanity.com/paper/2309.11751/) (2023)
- [A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation](https://mlsanity.com/paper/2305.11391/) (2023)
- [Exploiting LLM Quantization](https://mlsanity.com/paper/2405.18137/) (2024)
- [CHAMP: A Competition-level Dataset for Fine-Grained Analyses of LLMs' Mathematical Reasoning Capabilities](https://mlsanity.com/paper/2401.06961/) (2024)
- [An Empirical Study towards Characterizing Deep Learning Development and Deployment across Different Frameworks and Platforms](https://mlsanity.com/paper/1909.06727/) (2019)
- [Large Language Model Supply Chain: A Research Agenda](https://mlsanity.com/paper/2404.12736/) (2024)
- [A Unified Framework for Adversarial Attack and Defense in Constrained Feature Space](https://mlsanity.com/paper/2112.01156/) (2021)
- [Models Are Codes: Towards Measuring Malicious Code Poisoning Attacks on Pre-trained Model Hubs](https://mlsanity.com/paper/2409.09368/) (2024)
- [Evaluating the Robustness of Test Selection Methods for Deep Neural Networks](https://mlsanity.com/paper/2308.01314/) (2023)

## Built on by (most cited first, 9 of 9)

- [Bridging AI and Software Security: A Comparative Vulnerability Assessment of LLM Agent Deployment Paradigms](https://mlsanity.com/paper/2507.06323/) (2025)
- [Privacy Collapse: Benign Fine-Tuning Can Break Contextual Privacy in Language Models](https://mlsanity.com/paper/2601.15220/) (2026)
- [Causal-Guided Detoxify Backdoor Attack of Open-Weight LoRA Models](https://mlsanity.com/paper/2512.19297/) (2025)
- [Securing Educational LLMs: A Generalised Taxonomy of Attacks on LLMs and DREAD Risk Assessment](https://mlsanity.com/paper/2508.08629/) (2025)
- [Colluding LoRA: A Compositional Vulnerability in LLM Safety Alignment](https://mlsanity.com/paper/2603.12681/) (2026)
- [AgentFlow: Building Agent Dependency Graphs for Static Analysis of Agent Programs](https://mlsanity.com/paper/2607.01640/) (2026)
- [The Art of (Mis)alignment: How Fine-Tuning Methods Effectively Misalign and Realign LLMs in Post-Training](https://mlsanity.com/paper/2604.07754/) (2026)
- [From Adoption to Deployment: A Qualitative Study on AI Integration in Software Development Practice](https://mlsanity.com/paper/2607.16660/) (2026)
- [On Understanding, Identifying, and Mitigating Vulnerabilities in Agentic Large Language Models](https://mlsanity.com/paper/2608.10530/) (2026)

## Abstract (arXiv)

Large Language Model (LLM) is changing the software development paradigm and has gained huge attention from both academia and industry. Researchers and developers collaboratively explore how to leverage the powerful problem-solving ability of LLMs for specific domain tasks. Due to the wide usage of LLM-based applications, e.g., ChatGPT, multiple works have been proposed to ensure the security of LLM systems. However, a comprehensive understanding of the entire processes of LLM system construction (the LLM supply chain) is crucial but relevant works are limited. More importantly, the security issues hidden in the LLM SC which could highly impact the reliable usage of LLMs are lack of exploration. Existing works mainly focus on assuring the quality of LLM from the model level, security assurance for the entire LLM SC is ignored. In this work, we take the first step to discuss the potential security risks in each component as well as the integration between components of LLM SC. We summarize 12 security-related risks and provide promising guidance to help build safer LLM systems. We hope our work can facilitate the evolution of artificial general intelligence with secure LLM ecosystems.