mlsanityData to 7 Oct 2026

Identifying and Mitigating the Security Risks of Generative AI

arXiv 2308.14840 ↗First posted 28 Aug 2023cs.AIcited by 90

Not on the map. Why?

Cited by 32 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Every major technical invention resurfaces the dual-use dilemma -- the new technology has the potential to be used for good as well as for harm. Generative AI (GenAI) techniques, such as large language models (LLMs) and diffusion models, have shown remarkable capabilities (e.g., in-context learning, code-completion, and text-to-image generation and editing). However, GenAI can be used just as well by attackers to generate new attacks and increase the velocity and efficacy of existing attacks. This paper reports the findings of a workshop held at Google (co-organized by Stanford University and the University of Wisconsin-Madison) on the dual-use dilemma posed by GenAI. This paper is not meant to be comprehensive, but is rather an attempt to synthesize some of the interesting findings from the workshop. We discuss short-term and long-term goals for the community on this topic. We hope this paper provides both a launching point for a discussion on this important topic as well as interesting problems that the research community can work to address.

Key facts. Cited by 90 papers in the map (data through 7 Oct 2026). Builds on 72 of its 114 references that are in the map. First posted on arXiv 28 Aug 2023.

Builds on

72 of 114 references.

References · 72

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  6. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  7. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  8. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  9. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  10. 2020Learning to summarize from human feedbackcited by 2,562
  11. 2022Emergent Abilities of Large Language Modelscited by 2,483
  12. 2020Extracting Training Data from Large Language Modelscited by 2,144
38 more
  1. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  2. 2022Red Teaming Language Models with Language Modelscited by 983
  3. 2019CNN-generated images are surprisingly easy to spot... for nowcited by 842
  4. 2023DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvaturecited by 705
  5. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  6. 2023In-Context Retrieval-Augmented Language Modelscited by 694
  7. 2023A Watermark for Large Language Modelscited by 667
  8. 2023ChemCrow: Augmenting large-language models with chemistry toolscited by 609
  9. 2019Release Strategies and the Social Impacts of Language Modelscited by 505
  10. 2020Leveraging Frequency Analysis for Deep Fake Image Recognitioncited by 501
  11. 2019GLTR: Statistical Detection and Visualization of Generated Textcited by 490
  12. 2023Towards Universal Fake Image Detectors that Generalize Across Generative Modelscited by 483
  13. 2020Artificial Intelligence, Values and Alignmentcited by 443
  14. 2017Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraintscited by 420
  15. 2021Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributionscited by 417
  16. 2023Detecting Language Model Attacks with Perplexitycited by 411
  17. 2023Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defensecited by 401
  18. 2023Can AI-Generated Text be Reliably Detected?cited by 372
  19. 2023The Curse of Recursion: Training on Generated Data Makes Models Forgetcited by 343
  20. 2023Are aligned neural networks adversarially aligned?cited by 305
  21. 2019Detecting and Simulating Artifacts in GAN Fake Imagescited by 293
  22. 2022Teaching language models to support answers with verified quotescited by 290
  23. 2023The Stable Signature: Rooting Watermarks in Latent Diffusion Modelscited by 289
  24. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 286
  25. 2023Robust Distortion-free Watermarks for Language Modelscited by 279
  26. 2023Poisoning Web-Scale Training Datasets is Practicalcited by 276
  27. 2023Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attackscited by 268
  28. 2019Automatic Detection of Generated Text is Easiest when Humans are Fooledcited by 260
  29. 2023Provable Robust Watermarking for AI-Generated Textcited by 259
  30. 2021Measuring Attribution in Natural Language Generation Modelscited by 253
  31. 2023Can Large Language Models Transform Computational Social Science?cited by 245
  32. 2020Global Texture Enhancement for Fake Face Detection in the Wildcited by 220
  33. 2023PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Promptscited by 218
  34. 2023On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspectivecited by 208
  35. 2023Undetectable Watermarks for Language Modelscited by 208
  36. 2021Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasetscited by 179
  37. 2023GPT detectors are biased against non-native English writerscited by 175
  38. 2023On the Reliability of Watermarks for Large Language Modelscited by 170

50 most cited of 72 All 72 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 12; 2024: 41; 2025: 28; 2026: 9.

90 papers.

Sort:

By area:

  1. 2023LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Datasetcited by 449
  2. 2024Knowledge Conflicts for LLMs: A Surveycited by 274
  3. 2024Rethinking Machine Unlearning for Large Language Modelscited by 247
  4. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attackscited by 189
  5. 2023Can LLM-Generated Misinformation Be Detected?cited by 173
  6. 2023Removing RLHF Protections in GPT-4 via Fine-Tuningcited by 165
  7. 2023Raising the Bar of AI-generated Image Detection with CLIPcited by 163
  8. 2024SOUL: Unlocking the Power of Second-Order Optimization for LLM Unlearningcited by 104
  9. 2024Soft Prompt Threats: Attacking Safety Alignment and Unlearning in Open-Source LLMs through the Embedding Spacecited by 88
  10. 2024DeMamba: AI-Generated Video Detection on Million-Scale GenVideo Benchmarkcited by 85
  11. 2024LLM Unlearning via Loss Adjustment with Only Forget Datacited by 71
  12. 2024When LLMs Meet Cybersecurity: A Systematic Literature Reviewcited by 70
38 more
  1. 2024A Bias-Free Training Paradigm for More General AI-generated Image Detectioncited by 67
  2. 2024T2VSafetyBench: Evaluating the Safety of Text-to-Video Generative Modelscited by 66
  3. 2024On the Societal Impact of Open Foundation Modelscited by 64
  4. 2023Leveraging Optimization for Adaptive Attacks on Image Watermarkscited by 50
  5. 2024Self-Alignment of Large Language Models via Monopolylogue-based Social Scene Simulationcited by 46
  6. 2025Exploring Criteria of Loss Reweighting to Enhance LLM Unlearningcited by 39
  7. 2025LLM Unlearning via Neural Activation Redirectioncited by 38
  8. 2023Interpretable Diffusion via Information Decompositioncited by 37
  9. 2024CLEAR: Character Unlearning in Textual and Visual Modalitiescited by 36
  10. 2024Revisiting Who's Harry Potter: Towards Targeted Unlearning from a Causal Intervention Perspectivecited by 33
  11. 2025D3: Training-Free AI-Generated Video Detection Using Second-Order Featurescited by 32
  12. 2024How Far Are We From AGI: Are LLMs All We Need?cited by 31
  13. 2024Adversaries Can Misuse Combinations of Safe Modelscited by 31
  14. 2024MOSSBench: Is Your Multimodal Language Model Oversensitive to Safe Queries?cited by 28
  15. 2024Detecting AI-Generated Video via Frame Consistencycited by 26
  16. 2024WAGLE: Strategic Weight Attribution for Effective and Modular Unlearning in Large Language Modelscited by 25
  17. 2024Kernel PCA for Out-of-Distribution Detectioncited by 24
  18. 2025BLUR: A Bi-Level Optimization Approach for LLM Unlearningcited by 23
  19. 2025Seeing What Matters: Generalizable AI-generated Video Detection with Forensic-Oriented Augmentationcited by 23
  20. 2025Frontier AI's Impact on the Cybersecurity Landscapecited by 21
  21. 2023Can Large Language Models Provide Security & Privacy Advice? Measuring the Ability of LLMs to Refute Misconceptionscited by 20
  22. 2024Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Datacited by 20
  23. 2024Grounding and Evaluation for Large Language Models: Practical Challenges and Lessons Learned (Survey)cited by 20
  24. 2024Latent Watermark: Inject and Detect Watermarks in Latent Diffusion Spacecited by 19
  25. 2024Towards Robust Evaluation of Unlearning in LLMs via Data Transformationscited by 19
  26. 2024Unveiling Entity-Level Unlearning for Large Language Models: A Comprehensive Analysiscited by 14
  27. 2024UnStar: Unlearning with Self-Taught Anti-Sample Reasoning for LLMscited by 14
  28. 2024Synthetic Image Verification in the Era of Generative AI: What Works and What Isn't There Yetcited by 13
  29. 2024Legilimens: Practical and Unified Content Moderation for Large Language Model Servicescited by 13
  30. 2025Dialogue Injection Attack: Jailbreaking LLMs through Context Manipulationcited by 13
  31. 2024The Call for Socially Aware Language Technologiescited by 12
  32. 2024Optimizing Adaptive Attacks against Watermarks for Language Modelscited by 11
  33. 2023Safety Alignment in NLP Tasks: Weakly Aligned Summarization as an In-Context Attackcited by 10
  34. 2025AI-GenBench: A New Ongoing Benchmark for AI-Generated Image Detectioncited by 10
  35. 2025Unlearning That Lasts: Utility-Preserving, Robust, and Almost Irreversible Forgetting in LLMscited by 9
  36. 2025Leak@k: Unlearning Does Not Make LLMs Forget Under Probabilistic Decodingcited by 9
  37. 2025VeriStruct: AI-assisted Automated Verification of Data-Structure Modules in Veruscited by 8
  38. 2023Warfare:Breaking the Watermark Protection of AI-Generated Contentcited by 7

50 most cited of 90 All 90 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Computational Safety for Generative AI: A Hypothesis Testing Perspective
  2. 2026From Alignment to Access Control: A Framework for GenAI Policy Enforcement
  3. 2025Securing Generative AI Agentic Workflows: Risks, Mitigation, and a Proposed Firewall Architecture
  4. 2024Generative AI in Cybersecurity: A Comprehensive Review of LLM Applications and Vulnerabilities
  5. 2026Large Language Models (LLMs) and Generative AI in Cybersecurity and Privacy: A Survey of Dual-Use Risks, AI-Generated Malware, Explainability, and Defensive Strategies
  6. 2024AI Safety in Generative AI Large Language Models: A Survey
  7. 2025Securing Agentic AI: A Comprehensive Threat Model and Mitigation Framework for Generative AI Agents
  8. 2025GenBreak: Red Teaming Text-to-Image Generators Using Large Language Models