mlsanityData to 14 Oct 2026

Generative AI Security: Challenges and Countermeasures

arXiv 2402.12617 ↗First posted 20 Feb 2024cs.CR cs.AI cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

Generative AI's expanding footprint across numerous industries has led to both excitement and increased scrutiny. This paper delves into the unique security challenges posed by Generative AI, and outlines potential research directions for managing these risks.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 54 of its 80 references that are in the map. First posted on arXiv 20 Feb 2024.

Builds on

54 of 80 references.

References · 54

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  7. 2023Visual Instruction Tuningcited by 7,869
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  9. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  10. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  11. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  12. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
38 more
  1. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  2. 2020Learning to summarize from human feedbackcited by 2,588
  3. 2022Emergent Abilities of Large Language Modelscited by 2,512
  4. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  5. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  6. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  7. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
  8. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  9. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,114
  10. 2022Scaling Laws for Reward Model Overoptimizationcited by 1,095
  11. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 832
  12. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 729
  13. 2023A Watermark for Large Language Modelscited by 725
  14. 2023Prompt Injection attack against LLM-integrated Applicationscited by 605
  15. 2020On Adaptive Attacks to Adversarial Example Defensescited by 555
  16. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 498
  17. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 434
  18. 2021FUDGE: Controlled Text Generation With Future Discriminatorscited by 376
  19. 2022A Holistic Approach to Undesired Content Detection in the Real Worldcited by 356
  20. 2023Statistical Rejection Sampling Improves Preference Optimizationcited by 323
  21. 2023Robust Distortion-free Watermarks for Language Modelscited by 320
  22. 2023Provable Robust Watermarking for AI-Generated Textcited by 299
  23. 2019Detecting and Simulating Artifacts in GAN Fake Imagescited by 293
  24. 2023Poisoning Web-Scale Training Datasets is Practicalcited by 290
  25. 2023OpenChat: Advancing Open-source Language Models with Mixed-Quality Datacited by 252
  26. 2023Principled Reinforcement Learning with Human Feedback from Pairwise or K-wise Comparisonscited by 247
  27. 2023LLM Self Defense: By Self Examination, LLMs Know They Are Being Trickedcited by 244
  28. 2023Undetectable Watermarks for Language Modelscited by 226
  29. 2022COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamicscited by 161
  30. 2023Tensor Trust: Interpretable Prompt Injection Attacks from an Online Gamecited by 145
  31. 2023From ChatGPT to ThreatGPT: Impact of Generative AI in Cybersecurity and Privacycited by 143
  32. 2023Controlled Decoding from Language Modelscited by 135
  33. 2023Universal Jailbreak Backdoors from Poisoned Human Feedbackcited by 128
  34. 2023Levels of AGI for Operationalizing Progress on the Path to AGIcited by 96
  35. 2022Evaluating the Susceptibility of Pre-Trained Language Models via Handcrafted Adversarial Examplescited by 69
  36. 2022Is GitHub's Copilot as Bad as Humans at Introducing Vulnerabilities in Code?cited by 67
  37. 2023Unveiling Security, Privacy, and Ethical Concerns of ChatGPTcited by 53
  38. 2023LLM Platform Security: Applying a Systematic Evaluation Framework to OpenAI's ChatGPT Pluginscited by 51

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 3.

4 papers.

  1. 2025Towards Watermarking of Open-Source LLMscited by 13
  2. 2024SecGenAI: Enhancing Security of Cloud-based Generative AI Applications within Australian Critical Technologies of National Interestcited by 2
  3. 2025Practical, Generalizable and Robust Backdoor Attacks on Text-to-Image Diffusion Modelscited by 1
  4. 2025A Real-Time, Self-Tuning Moderator Framework for Adversarial Prompt Detectioncited by 0