mlsanityData to 7 Oct 2026

Malla: Demystifying Real-world Large Language Model Integrated Malicious Services

arXiv 2401.03315 ↗First posted 6 Jan 2024cs.CR cs.AIcited by 24

Not on the map. Why?

Abstract (arXiv)

The underground exploitation of large language models (LLMs) for malicious services (i.e., Malla) is witnessing an uptick, amplifying the cyber threat landscape and posing questions about the trustworthiness of LLM technologies. However, there has been little effort to understand this new cybercrime, in terms of its magnitude, impact, and techniques. In this paper, we conduct the first systematic study on 212 real-world Mallas, uncovering their proliferation in underground marketplaces and exposing their operational modalities. Our study discloses the Malla ecosystem, revealing its significant growth and impact on today's public LLM services. Through examining 212 Mallas, we uncovered eight backend LLMs used by Mallas, along with 182 prompts that circumvent the protective measures of public LLM APIs. We further demystify the tactics employed by Mallas, including the abuse of uncensored LLMs and the exploitation of public LLM APIs through jailbreak prompts. Our findings enable a better understanding of the real-world exploitation of LLMs by cybercriminals, offering insights into strategies to counteract this cybercrime.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Builds on 21 of its 83 references that are in the map. First posted on arXiv 6 Jan 2024.

Builds on

21 of 83 references.

References · 21

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  4. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  5. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  6. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  7. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
  8. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 478
  9. 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Modelscited by 457
  10. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 434
  11. 2023TinyStories: How Small Can Language Models Be and Still Speak Coherent English?cited by 425
  12. 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 347
9 more
  1. 2022A Holistic Approach to Undesired Content Detection in the Real Worldcited by 337
  2. 2018code2vec: Learning Distributed Representations of Codecited by 249
  3. 2023Defending Against Alignment-Breaking Attacks via Robustly Aligned LLMcited by 173
  4. 2023Unsafe Diffusion: On the Generation of Unsafe Images and Hateful Memes From Text-To-Image Modelscited by 171
  5. 2023From ChatGPT to ThreatGPT: Impact of Generative AI in Cybersecurity and Privacycited by 137
  6. 2022Evaluating the Susceptibility of Pre-Trained Language Models via Handcrafted Adversarial Examplescited by 65
  7. 2023Latent Jailbreak: A Benchmark for Evaluating Text Safety and Output Robustness of Large Language Modelscited by 56
  8. 2023Ethical Frameworks and Computer Security Trolley Problems: Foundations for Conversationscited by 10
  9. 2020Authorship Attribution of Source Code: A Language-Agnostic Approach and Applicability in Software Engineeringcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 11; 2025: 8; 2026: 5.

24 papers.

Sort:

By area:

  1. 2024When LLMs Meet Cybersecurity: A Systematic Literature Reviewcited by 70
  2. 2024Voice Jailbreak Attacks Against GPT-4ocited by 39
  3. 2024When LLMs Go Online: The Emerging Threat of Web-Enabled LLMscited by 22
  4. 2024Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Datacited by 20
  5. 2025PurpCode: Reasoning for Safer Code Generationcited by 17
  6. 2025Large Language Model (LLM) for Software Security: Code Analysis, Malware Analysis, Reverse Engineeringcited by 15
  7. 2025Dialogue Injection Attack: Jailbreaking LLMs through Context Manipulationcited by 13
  8. 2024On the (In)Security of LLM App Storescited by 12
  9. 2025Malicious LLM-Based Conversational AI Makes Users Reveal Personal Informationcited by 10
  10. 2024Locking Machine Learning Models into Hardwarecited by 6
  11. 2025Beyond Data Privacy: New Privacy Risks for Large Language Modelscited by 6
  12. 2024Are You Human? An Adversarial Benchmark to Expose LLMscited by 5
12 more
  1. 2025MOCHA: Are Code Language Models Robust Against Multi-Turn Malicious Coding Prompts?cited by 5
  2. 2025Paladin: Defending LLM-enabled Phishing Emails with a New Trigger-Tag Paradigmcited by 4
  3. 2026You Told Me to Do It: Measuring Instructional Text-induced Private Data Leakage in LLM Agentscited by 3
  4. 2024Code Vulnerability Repair with Large Language Model using Context-Aware Prompt Tuningcited by 2
  5. 2024Countering Autonomous Cyber Threatscited by 1
  6. 2024LLM App Squatting and Cloningcited by 1
  7. 2026Implicit Identity Technologies for LLMs: Fingerprinting and Watermarking across Datasets, Models, and Generated Contentcited by 1
  8. 2024Transforming Computer Security and Public Trust Through the Exploration of Fine-Tuning Large Language Modelscited by 0
  9. 2025MGC: A Compiler Framework Exploiting Compositional Blindness in Aligned LLMs for Malware Generationcited by 0
  10. 2026Stand-Alone Complex or Vibercrime? Exploring the adoption and innovation of GenAI tools, coding assistants, and agents within cybercrime ecosystemscited by 0
  11. 2026Uncensored Open-weight Models: Redistribution as the Persistence Layercited by 0
  12. 2026CS-Guard: Benchmarking LLM Guardrails for Code Generation Securitycited by 0