# Models Are Codes: Towards Measuring Malicious Code Poisoning Attacks on Pre-trained Model Hubs

arXiv: https://arxiv.org/abs/2409.09368 · HTML: https://mlsanity.com/paper/2409.09368/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Sep 2024.

## Built on by (most cited first, 18 of 18)

- [Large Language Model Supply Chain: Open Problems From the Security Perspective](https://mlsanity.com/paper/2411.01604/) (2024)
- [Enhancing the De-identification of Personally Identifiable Information in Educational Data](https://mlsanity.com/paper/2501.09765/) (2025)
- [PickleBall: Secure Deserialization of Pickle-based Machine Learning Models (Extended Report)](https://mlsanity.com/paper/2508.15987/) (2025)
- [A Rusty Link in the AI Supply Chain: Detecting Evil Configurations in Model Repositories](https://mlsanity.com/paper/2505.01067/) (2025)
- [Consiglieres in the Shadow: Understanding the Use of Uncensored Large Language Models in Cybercrimes](https://mlsanity.com/paper/2508.12622/) (2025)
- [On the (In)Security of Loading Machine Learning Models](https://mlsanity.com/paper/2509.06703/) (2025)
- [AgentRAE: Remote Action Execution through Notification-based Visual Backdoors against Screenshots-based Mobile GUI Agents](https://mlsanity.com/paper/2603.23007/) (2026)
- [Broken on Arrival: Silently Defective LLM Artifacts in Public Model Registries and How to Catch Them](https://mlsanity.com/paper/2609.05881/) (2026)
- [Living Off the LLM: How LLMs Will Change Adversary Tactics](https://mlsanity.com/paper/2510.11398/) (2025)
- [Securing the AI Supply Chain: What Can We Learn From Developer-Reported Security Issues and Solutions of AI Projects?](https://mlsanity.com/paper/2512.23385/) (2025)
- [SafePickle: Robust and Generic ML Detection of Malicious Pickle-based ML Models](https://mlsanity.com/paper/2602.19818/) (2026)
- [Cyber-Capable AI Agents: Vulnerabilities, Evaluation Containment, and Defensive Response](https://mlsanity.com/paper/2607.25379/) (2026)
- [Zero-Trust Artificial Intelligence Model Security Based on Moving Target Defense and Content Disarm and Reconstruction](https://mlsanity.com/paper/2503.01758/) (2025)
- [One RNG to Rule Them All: How Randomness Becomes an Attack Vector in Machine Learning](https://mlsanity.com/paper/2602.09182/) (2026)
- [BadDLM: Backdooring Diffusion Language Models with Diverse Targets](https://mlsanity.com/paper/2605.09397/) (2026)
- [Toward a Generalized Defense Across Sparse, Continuous, and Structured Parameter Attacks](https://mlsanity.com/paper/2606.04317/) (2026)
- [Lifecycle-Aware Dynamic Analysis for Secure ML Model Execution](https://mlsanity.com/paper/2606.19023/) (2026)
- [GrayShield: Bit-Level Sanitization for Transformer Model Supply-Chain Security](https://mlsanity.com/paper/2610.04319/) (2026)

## Abstract (arXiv)

The proliferation of pre-trained models (PTMs) and datasets has led to the emergence of centralized model hubs like Hugging Face, which facilitate collaborative development and reuse. However, recent security reports have uncovered vulnerabilities and instances of malicious attacks within these platforms, highlighting growing security concerns. This paper presents the first systematic study of malicious code poisoning attacks on pre-trained model hubs, focusing on the Hugging Face platform. We conduct a comprehensive threat analysis, develop a taxonomy of model formats, and perform root cause analysis of vulnerable formats. While existing tools like Fickling and ModelScan offer some protection, they face limitations in semantic-level analysis and comprehensive threat detection. To address these challenges, we propose MalHug, an end-to-end pipeline tailored for Hugging Face that combines dataset loading script extraction, model deserialization, in-depth taint analysis, and heuristic pattern matching to detect and classify malicious code poisoning attacks in datasets and models. In collaboration with Ant Group, a leading financial technology company, we have implemented and deployed MalHug on a mirrored Hugging Face instance within their infrastructure, where it has been operational for over three months. During this period, MalHug has monitored more than 705K models and 176K datasets, uncovering 91 malicious models and 9 malicious dataset loading scripts. These findings reveal a range of security threats, including reverse shell, browser credential theft, and system reconnaissance. This work not only bridges a critical gap in understanding the security of the PTM supply chain but also provides a practical, industry-tested solution for enhancing the security of pre-trained model hubs.