Your Space is My Zone: Demystifying the Security Risks of AI-Powered Applications on Pre-Trained Model Hubs
Not on the map. Why?
Abstract (arXiv)
AI-powered Applications (AI-Apps), hosted on platforms such as Hugging Face, are democratizing access to pre-trained models through online inference and fine-tuning services. While lowering AI adoption barriers, these platforms introduce an unexplored attack surface, as AI-Apps are often developed by untrusted parties with weak isolation and misconfigured security settings. In this paper, we present the first systematic security analysis of AI-Apps across three leading platforms. To structure our investigation, we map the AI-App lifecycle to established risk taxonomies (e.g., OWASP), identifying five threat categories and ten attack vectors ranging from generic web flaws to high-impact architectural issues. Our analysis reveals critical failures including broken access control, insecure resource reuse, insufficient input validation, and sensitive data exposure. Notably, we uncover three novel architectural vulnerabilities inherent to platform design and demonstrate how traditional issues (e.g., world-readable logs) are uniquely amplified in this ecosystem. To assess real-world impact, we develop an analysis framework Insightor and apply it to over 970,000 public AI-Apps. Alarmingly, we find thousands of apps leaking credentials, hundreds containing input injection vulnerabilities that allow arbitrary code execution, and tens harboring embedded backdoors -- indicating active exploitation. We have responsibly disclosed all findings to the affected platforms and developers.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). First posted on arXiv 29 Jun 2026.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2026: 1.
1 paper.
- 2026Code-Poisoning Property Inference Attackscited by 0
- 2026Code-Poisoning Property Inference Attacks17 Jul 2026
Similar, not cited
Close in wording, no citation either way
- 2025Beyond Model Jailbreak: Systematic Dissection of the "Ten DeadlySins" in Embodied Intelligence
- 2025A Systematization of Security Vulnerabilities in Computer Use Agents
- 2026AI Security in the Foundation Model Era: A Comprehensive Survey from a Unified Perspective
- 2025When AI Takes the Wheel: Security Analysis of Framework-Constrained Program Generation ↗
- 2026Your Agent, Their Asset: A Real-World Safety Analysis of OpenClaw
- 2026Agent Skills in the Wild: An Empirical Study of Security Vulnerabilities at Scale
- 2025"Abuse Risks are Often Inherent to Product Features": Exploring AI Vendors' Bug Bounty and Responsible Disclosure Policies
- 2026A Translational Note on AI Safety Evaluation