mlsanityData to 7 Oct 2026

A Large-Scale Exploit Instrumentation Study of AI/ML Supply Chain Attacks in Hugging Face Models

arXiv 2410.04490 ↗First posted 6 Oct 2024cs.CR cs.LG cs.SEcited by 12

Not on the map. Why?

Abstract (arXiv)

The development of machine learning (ML) techniques has led to ample opportunities for developers to develop and deploy their own models. Hugging Face serves as an open source platform where developers can share and download other models in an effort to make ML development more collaborative. In order for models to be shared, they first need to be serialized. Certain Python serialization methods are considered unsafe, as they are vulnerable to object injection. This paper investigates the pervasiveness of these unsafe serialization methods across Hugging Face, and demonstrates through an exploitation approach, that models using unsafe serialization methods can be exploited and shared, creating an unsafe environment for ML developers. We investigate to what extent Hugging Face is able to flag repositories and files using unsafe serialization methods, and develop a technique to detect malicious models. Our results show that Hugging Face is home to a wide range of potentially vulnerable models.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 23 of its 74 references that are in the map. First posted on arXiv 6 Oct 2024.

Builds on

23 of 74 references.

References · 23

  1. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  2. 2019Local Model Poisoning Attacks to Byzantine-Robust Federated Learningcited by 479
  3. 2020Weight Poisoning Attacks on Pre-trained Modelscited by 318
  4. 2021Membership Inference Attacks on Machine Learning: A Surveycited by 306
  5. 2020Trojaning Language Models for Fun and Profitcited by 80
  6. 2019Prediction Poisoning: Towards Defenses Against DNN Model Stealing Attackscited by 79
  7. 2020Code Prediction by Feeding Trees to Transformerscited by 70
  8. 2022Taxonomy of Attacks on Open-Source Software Supply Chainscited by 57
  9. 2022CodeFill: Multi-token Code Completion by Jointly Learning from Structure and Naming Sequencescited by 49
  10. 2023An Empirical Study of Pre-Trained Model Reuse in the Hugging Face Deep Learning Model Registrycited by 45
  11. 2021DeepPayload: Black-box Backdoor Attack on Deep Learning Models through Neural Payload Injectioncited by 42
  12. 2024Breaking Down the Defenses: A Comparative Survey of Attacks on Large Language Modelscited by 42
11 more
  1. 2020Fast and Memory-Efficient Neural Code Completioncited by 38
  2. 2022Deep Learning Meets Software Engineering: A Survey on Pre-Trained Models of Source Codecited by 20
  3. 2020Compiling ONNX Neural Network Models Using MLIRcited by 18
  4. 2022M2TS: Multi-Scale Multi-Modal Approach Based on Transformer for Source Code Summarizationcited by 14
  5. 2024DONAPI: Malicious NPM Packages Detector using Behavior Sequence Knowledge Mappingcited by 6
  6. 2023ODDFUZZ: Discovering Java Deserialization Vulnerabilities via Structure-Aware Directed Greybox Fuzzingcited by 4
  7. 2024MalModel: Hiding Malicious Payload in Mobile Deep Learning Models with Black-box Backdoor Attackcited by 4
  8. 2024ONNXPruner: ONNX-Based General Model Pruning Adaptercited by 3
  9. 2022An In-depth Study of Java Deserialization Remote-Code Execution Exploits and Vulnerabilitiescited by 2
  10. 2023Seneca: Taint-Based Call Graph Construction for Java Object Deserializationcited by 2
  11. 2023Improving Java Deserialization Gadget Chain Mining via Overriding-Guided Object Generationcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 8.

12 papers.

  1. 2025Fox in the Henhouse: Supply-Chain Backdoor Attacks Against Reinforcement Learningcited by 5
  2. 2025PickleBall: Secure Deserialization of Pickle-based Machine Learning Models (Extended Report)cited by 3
  3. 2025On the (In)Security of Loading Machine Learning Modelscited by 2
  4. 2025Securing the AI Supply Chain: What Can We Learn From Developer-Reported Security Issues and Solutions of AI Projects?cited by 1
  5. 2026Toward Reliable, Safe, and Secure LLMs for Scientific Applicationscited by 1
  6. 2026Inference-Time Backdoors via Chat Templates: From LLM Supply Chains to Agentic System Compromisecited by 0
  7. 2026When Prompts Become Payloads: A Framework for Mitigating SQL Injection Attacks in Large Language Model-Driven Applicationscited by 0
  8. 2026Lifecycle-Aware Dynamic Analysis for Secure ML Model Executioncited by 0
  9. 2026Cyber-Capable AI Agents: Vulnerabilities, Evaluation Containment, and Defensive Responsecited by 0
  10. 2026A Security-Oriented Lifecycle Model for Large Language Model Systemscited by 0
  11. 2026Broken on Arrival: Silently Defective LLM Artifacts in Public Model Registries and How to Catch Themcited by 0
  12. 2026SINGED: Correct Outputs Do Not Certify Safe Execution in LLM Agentscited by 0