mlsanityData to 7 Oct 2026

Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilities

arXiv 2308.12833 ↗First posted 24 Aug 2023cs.CL cs.CRcited by 76

Not on the map. Why?

Cited by 32 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Spurred by the recent rapid increase in the development and distribution of large language models (LLMs) across industry and academia, much recent work has drawn attention to safety- and security-related threats and vulnerabilities of LLMs, including in the context of potentially criminal activities. Specifically, it has been shown that LLMs can be misused for fraud, impersonation, and the generation of malware; while other authors have considered the more general problem of AI alignment. It is important that developers and practitioners alike are aware of security-related problems with such models. In this paper, we provide an overview of existing - predominantly scientific - efforts on identifying and mitigating threats and vulnerabilities arising from LLMs. We present a taxonomy describing the relationship between threats caused by the generative capabilities of LLMs, prevention measures intended to address such threats, and vulnerabilities arising from imperfect prevention measures. With our work, we hope to raise awareness of the limitations of LLMs in light of such security concerns, among both experienced developers and novel users of such technologies.

Key facts. Cited by 76 papers in the map (data through 7 Oct 2026). Builds on 157 of its 222 references that are in the map. First posted on arXiv 24 Aug 2023.

Builds on

157 of 222 references.

References · 157

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2023GPT-4 Technical Reportcited by 18,132
  6. 2022Training language models to follow instructions with human feedbackcited by 15,712
  7. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  8. 2017Proximal Policy Optimization Algorithmscited by 14,441
  9. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  10. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  11. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  12. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
38 more
  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  6. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  7. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  9. 2013Intriguing properties of neural networkscited by 5,114
  10. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  11. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  12. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  13. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  14. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  15. 2017Deep reinforcement learning from human preferencescited by 3,846
  16. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  17. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  18. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  19. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  20. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  21. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  22. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  23. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  24. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  25. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  26. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  27. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  28. 2020Learning to summarize from human feedbackcited by 2,562
  29. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  30. 2020Extracting Training Data from Large Language Modelscited by 2,144
  31. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  32. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  33. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  34. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  35. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  36. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  37. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  38. 2023A Survey on Evaluation of Large Language Modelscited by 1,227

50 most cited of 157 All 157 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 10; 2024: 40; 2025: 18; 2026: 8.

76 papers.

Sort:

By area:

  1. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 533
  2. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 380
  3. 2024A StrongREJECT for Empty Jailbreakscited by 352
  4. 2024ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMscited by 197
  5. 2024Security and Privacy Challenges of Large Language Models: A Surveycited by 179
  6. 2023Can LLM-Generated Misinformation Be Detected?cited by 173
  7. 2023Can Sensitive Information Be Deleted From LLMs? Objectives for Defending Against Extraction Attackscited by 156
  8. 2024Attacks, Defenses and Evaluations for LLM Conversation Safety: A Surveycited by 124
  9. 2024DrAttack: Prompt Decomposition and Reconstruction Makes Powerful LLM Jailbreakerscited by 91
  10. 2024Building Guardrails for Large Language Modelscited by 61
  11. 2024Introducing v0.5 of the AI Safety Benchmark from MLCommonscited by 60
  12. 2024Pretraining Data Detection for Large Language Models: A Divergence-based Calibration Methodcited by 54
38 more
  1. 2023Self-Guard: Empower the LLM to Safeguard Itselfcited by 52
  2. 2023Privacy in Large Language Models: Attacks, Defenses and Future Directionscited by 48
  3. 2025LLMs in Software Security: A Survey of Vulnerability Detection Techniques and Insightscited by 44
  4. 2023GenAI Against Humanity: Nefarious Applications of Generative Artificial Intelligence and Large Language Modelscited by 40
  5. 2023SoK: Memorization in General-Purpose Large Language Modelscited by 36
  6. 2024Securing Large Language Models: Threats, Vulnerabilities and Responsible Practicescited by 36
  7. 2023Advancements in Generative AI: A Comprehensive Review of GANs, GPT, Autoencoders, Diffusion Model, and Transformerscited by 35
  8. 2024An Early Categorization of Prompt Injection Attacks on Large Language Modelscited by 28
  9. 2024Dissecting Fine-Tuning Unlearning in Large Language Modelscited by 25
  10. 2025SafeDialBench: A Fine-Grained Safety Evaluation Benchmark for Large Language Models in Multi-Turn Dialogues with Diverse Jailbreak Attackscited by 23
  11. 2024Con-ReCall: Detecting Pre-training Data in LLMs via Contrastive Decodingcited by 20
  12. 2024Mapping the Ethics of Generative AI: A Comprehensive Scoping Reviewcited by 19
  13. 2023User Modeling in the Era of Large Language Models: Current Research and Future Directionscited by 18
  14. 2024Raccoon: Prompt Extraction Benchmark of LLM-Integrated Applicationscited by 16
  15. 2024MEOW: MEMOry Supervised LLM Unlearning Via Inverted Factscited by 16
  16. 2024On Catastrophic Inheritance of Large Foundation Modelscited by 15
  17. 2024Decoding the AI Pen: Techniques and Challenges in Detecting AI-Generated Textcited by 14
  18. 2024Detecting Training Data of Large Language Models via Expectation Maximizationcited by 13
  19. 2025LoX: Low-Rank Extrapolation Robustifies LLM Safety Against Fine-tuningcited by 12
  20. 2024AI Alignment through Reinforcement Learning from Human Feedback? Contradictions and Limitationscited by 10
  21. 2024Adaptive Pre-training Data Detection for Large Language Models via Surprising Tokenscited by 10
  22. 2025Does Localization Inform Unlearning? A Rigorous Examination of Local Parameter Attribution for Knowledge Unlearning in Language Modelscited by 10
  23. 2024HarmLevelBench: Evaluating Harm-Level Compliance and the Impact of Quantization on Model Alignmentcited by 9
  24. 2025GuidedBench: Measuring and Mitigating the Evaluation Discrepancies of In-the-wild LLM Jailbreak Methodscited by 9
  25. 2024A Survey on Responsible Generative AI: What to Generate and What Notcited by 8
  26. 2024SLM as Guardian: Pioneering AI Safety with Small Language Modelscited by 7
  27. 2024Fine-tuning can Help Detect Pretraining Data from Large Language Modelscited by 7
  28. 2024SMILES-Prompting: A Novel Approach to LLM Jailbreak Attacks in Chemical Synthesiscited by 7
  29. 2024Rethinking Software Engineering in the Foundation Model Era: A Curated Catalogue of Challenges in the Development of Trustworthy FMwarecited by 6
  30. 2025Code Red! On the Harmfulness of Applying Off-the-shelf Large Language Models to Programming Taskscited by 6
  31. 2026HoneyTrap: Deceiving Large Language Model Attackers to Honeypot Traps with Resilient Multi-Agent Defensecited by 6
  32. 2024Can LLMs be Fooled? Investigating Vulnerabilities in LLMscited by 5
  33. 2024SoK: Prompt Hacking of Large Language Modelscited by 5
  34. 2025Jailbreaking LLMs' Safeguard with Universal Magic Words for Text Embedding Modelscited by 5
  35. 2025Extracting and Understanding the Superficial Knowledge in Alignmentcited by 5
  36. 2024Optimizing RAG Techniques for Automotive Industry PDF Chatbots: A Case Study with Locally Deployed Ollama Modelscited by 4
  37. 2025FDLLM: A Dedicated Detector for Black-Box LLMs Fingerprintingcited by 4
  38. 2024Generative Models are Self-Watermarked: Declaring Model Authentication through Re-Generationcited by 3

50 most cited of 76 All 76 (JSON)