mlsanityData to 7 Oct 2026

AI Safety in Generative AI Large Language Models: A Survey

arXiv 2407.18369 ↗First posted 6 Jul 2024cs.CY cs.CLcited by 31

Not on the map. Why?

Abstract (arXiv)

Large Language Model (LLMs) such as ChatGPT that exhibit generative AI capabilities are facing accelerated adoption and innovation. The increased presence of Generative AI (GAI) inevitably raises concerns about the risks and safety associated with these models. This article provides an up-to-date survey of recent trends in AI safety research of GAI-LLMs from a computer scientist's perspective: specific and technical. In this survey, we explore the background and motivation for the identified harms and risks in the context of LLMs being generative language models; our survey differentiates by emphasising the need for unified theories of the distinct safety challenges in the research development and applications of LLMs. We start our discussion with a concise introduction to the workings of LLMs, supported by relevant literature. Then we discuss earlier research that has pointed out the fundamental constraints of generative models, or lack of understanding thereof (e.g., performance and safety trade-offs as LLMs scale in number of parameters). We provide a sufficient coverage of LLM alignment -- delving into various approaches, contending methods and present challenges associated with aligning LLMs with human preferences. By highlighting the gaps in the literature and possible implementation oversights, our aim is to create a comprehensive analysis that provides insights for addressing AI safety in LLMs and encourages the development of aligned and secure models. We conclude our survey by discussing future directions of LLMs for AI safety, offering insights into ongoing research in this critical area.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Builds on 182 of its 254 references that are in the map. First posted on arXiv 6 Jul 2024.

Builds on

182 of 254 references.

References · 182

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2017Proximal Policy Optimization Algorithmscited by 14,441
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  7. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  8. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  9. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  10. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  11. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  12. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
38 more
  1. 2020Scaling Laws for Neural Language Modelscited by 6,411
  2. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  4. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  7. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  8. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  9. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  10. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  11. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  12. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  13. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  14. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  15. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  16. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  17. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  18. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  19. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  20. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  21. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  22. 2020Learning to summarize from human feedbackcited by 2,562
  23. 2015Trust Region Policy Optimizationcited by 2,503
  24. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  25. 2020Extracting Training Data from Large Language Modelscited by 2,144
  26. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  27. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
  28. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  29. 2020Big Bird: Transformers for Longer Sequencescited by 1,673
  30. 2016Concrete Problems in AI Safetycited by 1,582
  31. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
  32. 2020Linformer: Self-Attention with Linear Complexitycited by 1,462
  33. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  34. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  35. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  36. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  37. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  38. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122

50 most cited of 182 All 182 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 8; 2025: 18; 2026: 5.

31 papers.

Sort:

By area:

  1. 2025PolyGuard: A Multilingual Safety Moderation Tool for 17 Languagescited by 57
  2. 2025A Comprehensive Survey of Reward Models: Taxonomy, Applications, Challenges, and Futurecited by 53
  3. 2025Agentic Web: Weaving the Next Web with AI Agentscited by 43
  4. 2024LLMs are Also Effective Embedding Models: An In-depth Overviewcited by 32
  5. 2025Oyster-I: Beyond Refusal -- Constructive Safety Alignment for Responsible Language Modelscited by 19
  6. 2024SciSafeEval: A Comprehensive Benchmark for Safety Alignment of Large Language Models in Scientific Taskscited by 17
  7. 2025Compromising Honesty and Harmlessness in Language Models via Deception Attackscited by 17
  8. 2025GUARD: Generation-time LLM Unlearning via Adaptive Restriction and Detectioncited by 17
  9. 2025Pre-Trained Policy Discriminators are General Reward Modelscited by 14
  10. 2025DrunkAgent: Stealthy Memory Corruption in LLM-Powered Recommender Agentscited by 12
  11. 2025JailDAM: Jailbreak Detection with Adaptive Memory for Vision-Language Modelcited by 10
  12. 2024PII-Scope: A Comprehensive Study on Training Data PII Extraction Attacks in LLMscited by 6
19 more
  1. 2024Recent Advances in Attack and Defense Approaches of Large Language Modelscited by 5
  2. 2026State Contamination in Memory-Augmented LLM Agentscited by 4
  3. 2025On the Inevitability of Left-Leaning Political Bias in Aligned Language Modelscited by 3
  4. 2025Retrieval-Augmented Review Generation for Poisoning Recommender Systemscited by 3
  5. 2025Probing Preference Representations: A Multi-Dimensional Evaluation and Analysis Method for Reward Modelscited by 3
  6. 2024Gen-AI for User Safety: A Surveycited by 2
  7. 2025Contextual Memory Intelligence -- A Foundational Paradigm for Human-AI Collaboration and Reflective Generative AI Systemscited by 2
  8. 2026The Hidden Puppet Master: Predicting Human Belief Change in Manipulative LLM Dialoguescited by 2
  9. 2024Bridging Today and the Future of Humanity: AI Safety in 2024 and Beyondcited by 1
  10. 2025LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systemscited by 1
  11. 2025From Symptoms to Systems: An Expert-Guided Approach to Understanding Risks of Generative AI for Eating Disorderscited by 1
  12. 2024A Formal Framework for Assessing and Mitigating Emergent Security Risks in Generative AI Models: Bridging Theory and Dynamic Risk Mitigationcited by 0
  13. 2024"Moralized" Multi-Step Jailbreak Prompts: Black-Box Testing of Guardrails in Large Language Models for Verbal Attackscited by 0
  14. 2025FLAME: Flexible LLM-Assisted Moderation Enginecited by 0
  15. 2025Computational Safety for Generative AI: A Hypothesis Testing Perspectivecited by 0
  16. 2025Limitations on Accurate, Trusted, Human-level Reasoningcited by 0
  17. 2026Responsible AI for General-Purpose Systems: Overview, Challenges, and A Path Forwardcited by 0
  18. 2026BarrierSteer: LLM Safety via Learning Barrier Steeringcited by 0
  19. 2026The safety failures we are not instrumenting: a perspective on hidden safety-critical challenges in modern AI systemscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Large Language Model Safety: A Holistic Survey
  2. 2024SG-Bench: Evaluating LLM Safety Generalization Across Diverse Tasks and Prompt Types
  3. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment
  4. 2024AEGIS: Online Adaptive AI Content Safety Moderation with Ensemble of LLM Experts
  5. 2024Concept-Guided LLM Agents for Human-AI Safety Codesign
  6. 2024Current state of LLM Risks and AI Guardrails
  7. 2025Security Concerns for Large Language Models: A Survey
  8. 2025Challenges and Applications of Large Language Models: A Comparison of GPT and DeepSeek family of models