# AI Safety in Generative AI Large Language Models: A Survey

arXiv: https://arxiv.org/abs/2407.18369 · HTML: https://mlsanity.com/paper/2407.18369/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 182 of its 254 references that are in the map.
- First posted on arXiv 6 Jul 2024.

## Builds on (most cited first, 30 of 182)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)

## Built on by (most cited first, 30 of 33)

- [Large Language Model Safety: A Holistic Survey](https://mlsanity.com/paper/2412.17686/) (2024)
- [PolyGuard: A Multilingual Safety Moderation Tool for 17 Languages](https://mlsanity.com/paper/2504.04377/) (2025)
- [A Comprehensive Survey of Reward Models: Taxonomy, Applications, Challenges, and Future](https://mlsanity.com/paper/2504.12328/) (2025)
- [Agentic Web: Weaving the Next Web with AI Agents](https://mlsanity.com/paper/2507.21206/) (2025)
- [LLMs are Also Effective Embedding Models: An In-depth Overview](https://mlsanity.com/paper/2412.12591/) (2024)
- [Oyster-I: Beyond Refusal -- Constructive Safety Alignment for Responsible Language Models](https://mlsanity.com/paper/2509.01909/) (2025)
- [SciSafeEval: A Comprehensive Benchmark for Safety Alignment of Large Language Models in Scientific Tasks](https://mlsanity.com/paper/2410.03769/) (2024)
- [Compromising Honesty and Harmlessness in Language Models via Deception Attacks](https://mlsanity.com/paper/2502.08301/) (2025)
- [GUARD: Generation-time LLM Unlearning via Adaptive Restriction and Detection](https://mlsanity.com/paper/2505.13312/) (2025)
- [Pre-Trained Policy Discriminators are General Reward Models](https://mlsanity.com/paper/2507.05197/) (2025)
- [DrunkAgent: Stealthy Memory Corruption in LLM-Powered Recommender Agents](https://mlsanity.com/paper/2503.23804/) (2025)
- [JailDAM: Jailbreak Detection with Adaptive Memory for Vision-Language Model](https://mlsanity.com/paper/2504.03770/) (2025)
- [PII-Scope: A Comprehensive Study on Training Data PII Extraction Attacks in LLMs](https://mlsanity.com/paper/2410.06704/) (2024)
- [Recent Advances in Attack and Defense Approaches of Large Language Models](https://mlsanity.com/paper/2409.03274/) (2024)
- [Generative AI for Biosciences: Emerging Threats and Roadmap to Biosecurity](https://mlsanity.com/paper/2510.15975/) (2025)
- [State Contamination in Memory-Augmented LLM Agents](https://mlsanity.com/paper/2605.16746/) (2026)
- [On the Inevitability of Left-Leaning Political Bias in Aligned Language Models](https://mlsanity.com/paper/2507.15328/) (2025)
- [Retrieval-Augmented Review Generation for Poisoning Recommender Systems](https://mlsanity.com/paper/2508.15252/) (2025)
- [Probing Preference Representations: A Multi-Dimensional Evaluation and Analysis Method for Reward Models](https://mlsanity.com/paper/2511.12464/) (2025)
- [Gen-AI for User Safety: A Survey](https://mlsanity.com/paper/2411.06606/) (2024)
- [Contextual Memory Intelligence -- A Foundational Paradigm for Human-AI Collaboration and Reflective Generative AI Systems](https://mlsanity.com/paper/2506.05370/) (2025)
- [The Hidden Puppet Master: Predicting Human Belief Change in Manipulative LLM Dialogues](https://mlsanity.com/paper/2603.20907/) (2026)
- [Bridging Today and the Future of Humanity: AI Safety in 2024 and Beyond](https://mlsanity.com/paper/2410.18114/) (2024)
- [LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systems](https://mlsanity.com/paper/2509.10682/) (2025)
- [From Symptoms to Systems: An Expert-Guided Approach to Understanding Risks of Generative AI for Eating Disorders](https://mlsanity.com/paper/2512.04843/) (2025)
- [A Formal Framework for Assessing and Mitigating Emergent Security Risks in Generative AI Models: Bridging Theory and Dynamic Risk Mitigation](https://mlsanity.com/paper/2410.13897/) (2024)
- ["Moralized" Multi-Step Jailbreak Prompts: Black-Box Testing of Guardrails in Large Language Models for Verbal Attacks](https://mlsanity.com/paper/2411.16730/) (2024)
- [FLAME: Flexible LLM-Assisted Moderation Engine](https://mlsanity.com/paper/2502.09175/) (2025)
- [Computational Safety for Generative AI: A Hypothesis Testing Perspective](https://mlsanity.com/paper/2502.12445/) (2025)
- [Limitations on Accurate, Trusted, Human-level Reasoning](https://mlsanity.com/paper/2509.21654/) (2025)

## Abstract (arXiv)

Large Language Model (LLMs) such as ChatGPT that exhibit generative AI capabilities are facing accelerated adoption and innovation. The increased presence of Generative AI (GAI) inevitably raises concerns about the risks and safety associated with these models. This article provides an up-to-date survey of recent trends in AI safety research of GAI-LLMs from a computer scientist's perspective: specific and technical. In this survey, we explore the background and motivation for the identified harms and risks in the context of LLMs being generative language models; our survey differentiates by emphasising the need for unified theories of the distinct safety challenges in the research development and applications of LLMs. We start our discussion with a concise introduction to the workings of LLMs, supported by relevant literature. Then we discuss earlier research that has pointed out the fundamental constraints of generative models, or lack of understanding thereof (e.g., performance and safety trade-offs as LLMs scale in number of parameters). We provide a sufficient coverage of LLM alignment -- delving into various approaches, contending methods and present challenges associated with aligning LLMs with human preferences. By highlighting the gaps in the literature and possible implementation oversights, our aim is to create a comprehensive analysis that provides insights for addressing AI safety in LLMs and encourages the development of aligned and secure models. We conclude our survey by discussing future directions of LLMs for AI safety, offering insights into ongoing research in this critical area.