# A Security-Oriented Lifecycle Model for Large Language Model Systems

arXiv: https://arxiv.org/abs/2608.03626 · HTML: https://mlsanity.com/paper/2608.03626/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 49 references that are in the map.
- First posted on arXiv 4 Aug 2026.

## Builds on (most cited first, 25 of 25)

- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://mlsanity.com/paper/2302.12173/) (2023)
- [Membership Inference Attacks on Machine Learning: A Survey](https://mlsanity.com/paper/2103.07853/) (2021)
- [Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directions](https://mlsanity.com/paper/2503.23278/) (2025)
- [The Emerged Security and Privacy of LLM Agent: A Survey with Case Studies](https://mlsanity.com/paper/2407.19354/) (2024)
- [Machine Learning Operations (MLOps): Overview, Definition, and Architecture](https://mlsanity.com/paper/2205.02302/) (2022)
- [Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoning](https://mlsanity.com/paper/2205.01992/) (2022)
- [RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs](https://mlsanity.com/paper/2404.08555/) (2024)
- [Exploiting LLM Quantization](https://mlsanity.com/paper/2405.18137/) (2024)
- [BadMerging: Backdoor Attacks Against Model Merging](https://mlsanity.com/paper/2408.07362/) (2024)
- [RLHFPoison: Reward Poisoning Attack for Reinforcement Learning with Human Feedback in Large Language Models](https://mlsanity.com/paper/2311.09641/) (2023)
- [Large Language Model Supply Chain: A Research Agenda](https://mlsanity.com/paper/2404.12736/) (2024)
- [The Pipeline for the Continuous Development of Artificial Intelligence Models -- Current State of Research and Practice](https://mlsanity.com/paper/2301.09001/) (2023)
- [Preference Poisoning Attacks on Reward Model Learning](https://mlsanity.com/paper/2402.01920/) (2024)
- [A Large-Scale Exploit Instrumentation Study of AI/ML Supply Chain Attacks in Hugging Face Models](https://mlsanity.com/paper/2410.04490/) (2024)
- [MoEcho: Exploiting Side-Channel Attacks to Compromise User Privacy in Mixture-of-Experts LLMs](https://mlsanity.com/paper/2508.15036/) (2025)
- [Analysis of Label-Flip Poisoning Attack on Machine Learning Based Malware Detector](https://mlsanity.com/paper/2301.01044/) (2023)
- [An Empirical Analysis of Machine Learning Model and Dataset Documentation, Supply Chain, and Licensing Challenges on Hugging Face](https://mlsanity.com/paper/2502.04484/) (2025)
- [AIBoMGen: Generating an AI Bill of Materials for Secure, Transparent, and Compliant Model Training](https://mlsanity.com/paper/2601.05703/) (2026)
- [Lifting the Veil on Composition, Risks, and Mitigations of the Large Language Model Supply Chain](https://mlsanity.com/paper/2410.21218/) (2024)
- [Cost-Minimized Label-Flipping Poisoning Attack to LLM Alignment](https://mlsanity.com/paper/2511.09105/) (2025)
- [Towards Secure MLOps: Surveying Attacks, Mitigation Strategies, and Research Challenges](https://mlsanity.com/paper/2506.02032/) (2025)
- [MetaBreak: Jailbreaking Online LLM Services via Special Token Manipulation](https://mlsanity.com/paper/2510.10271/) (2025)
- [RE for AI in Practice: Managing Data Annotation Requirements for AI Autonomous Driving Systems](https://mlsanity.com/paper/2511.15859/) (2025)
- [Securing the AI Supply Chain: What Can We Learn From Developer-Reported Security Issues and Solutions of AI Projects?](https://mlsanity.com/paper/2512.23385/) (2025)

## Abstract (arXiv)

Large language models are being integrated into critical infrastructure and enterprise workflows at unprecedented scale,yet the lifecycle frameworks governing their development and operations were designed for operational efficiency rather than security analysis. As a result, security-relevant activities such as data provenance verification, artifact signing, agentic permission control, and decommissioning are often left implicit or assumed to receive due care. Governance frameworks, in turn, organise requirements around risk levels or management processes without clearly linking them to the lifecycle stages where they apply. This paper addresses both deficiencies. We propose a lifecycle model for LLM systems that supports security analysis by structuring it around security-relevant boundaries rather than workflow optimisation. The model comprises 32 stages across four core pipeline layers (Data, Model, Distribution, Application), supported by a 12-stage LLMOps pillar and a 9-category governance pillar. Thirteen stages are introduced here as separate units because they expose distinct security concerns that existing frameworks do not clearly distinguish. A governance mapping synthesising the NIST AI RMF, the EU AI Act, and ISO/IEC 42001 reveals a structural property of the current regulatory landscape: governance evidence concentrates at deployment-facing stages, where systems are visible to regulators, while the most consequential decisions, data selection, alignment strategy, and capability boundaries, are made at development-facing stages, where regulatory visibility is lowest.