mlsanityData to 7 Oct 2026

ShieldGemma: Generative AI Content Moderation Based on Gemma

arXiv 2407.21772 ↗First posted 31 Jul 2024cs.CL cs.LGcited by 224

Not on the map. Why?

Cited by 126 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present ShieldGemma, a comprehensive suite of LLM-based safety content moderation models built upon Gemma2. These models provide robust, state-of-the-art predictions of safety risks across key harm types (sexually explicit, dangerous content, harassment, hate speech) in both user input and LLM-generated output. By evaluating on both public and internal benchmarks, we demonstrate superior performance compared to existing models, such as Llama Guard (+10.8% AU-PRC on public benchmarks) and WildCard (+4.3%). Additionally, we present a novel LLM-based data curation pipeline, adaptable to a variety of safety-related tasks and beyond. We have shown strong generalization performance for model trained mainly on synthetic data. By releasing ShieldGemma, we provide a valuable resource to the research community, advancing LLM safety and enabling the creation of more effective content moderation solutions for developers.

Key facts. Cited by 224 papers in the map (data through 7 Oct 2026). Builds on 25 of its 32 references that are in the map. First posted on arXiv 31 Jul 2024.

Builds on

25 of 32 references.

References · 25

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  5. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  6. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  7. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 828
  8. 2017Counterfactual Fairnesscited by 802
  9. 2019Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Boundscited by 602
  10. 2024WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMscited by 460
  11. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 415
  12. 2022A Holistic Approach to Undesired Content Detection in the Real Worldcited by 337
13 more
  1. 2024Humans or LLMs as the Judge? A Study on Judgement Biasescited by 314
  2. 2024On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Surveycited by 240
  3. 2024SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Modelscited by 225
  4. 2023ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI Conversationcited by 224
  5. 2022"I'm sorry to hear that": Finding New Biases in Language Models with a Holistic Descriptor Datasetcited by 160
  6. 2021Batch Active Learning at Scalecited by 127
  7. 2023Harnessing large-language models to generate private synthetic textcited by 65
  8. 2023AART: AI-Assisted Red-Teaming with Diverse Data Generation for New LLM-powered Applicationscited by 53
  9. 2022Data Augmentation for Intent Classification with Off-the-shelf Large Language Modelscited by 49
  10. 2022Self-Guided Noise-Free Data Generation for Efficient Zero-Shot Learningcited by 49
  11. 2024From LLM to Conversational Agent: A Memory Enhanced Architecture with Fine-Tuning of Large Language Modelscited by 42
  12. 2017Mix-and-Match Tuning for Self-Supervised Semantic Segmentationcited by 17
  13. 2022Ask Me What You Need: Product Retrieval using Knowledge from GPT-3cited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 15; 2025: 93; 2026: 116.

224 papers.

Sort:

By area:

  1. 2024SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformerscited by 331
  2. 2025SANA 1.5: Efficient Scaling of Training-Time and Inference-Time Compute in Linear Diffusion Transformercited by 165
  3. 2025Qwen3Guard Technical Reportcited by 125
  4. 2024Llama Guard 3 Vision: Safeguarding Human-AI Image Understanding Conversationscited by 118
  5. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  6. 2025GuardReasoner: Towards Reasoning-based LLM Safeguardscited by 89
  7. 2026ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedbackcited by 64
  8. 2025Safety in Large Reasoning Models: A Surveycited by 58
  9. 2025PolyGuard: A Multilingual Safety Moderation Tool for 17 Languagescited by 57
  10. 2025Detecting High-Stakes Interactions with Activation Probescited by 45
  11. 2024JAILJUDGE: A Comprehensive Jailbreak Judge Benchmark with Multi-Agent Enhanced Explanation Evaluation Frameworkcited by 44
  12. 2025Improving LLM Safety Alignment with Dual-Objective Optimizationcited by 40
38 more
  1. 2025GuardReasoner-VL: Safeguarding VLMs via Reinforced Reasoningcited by 38
  2. 2025Neither Valid nor Reliable? Investigating the Use of LLMs as Judgescited by 37
  3. 2024Automatic Pseudo-Harmful Prompt Generation for Evaluating False Refusals in Large Language Modelscited by 35
  4. 2025Enforcing Temporal Constraints for LLM Agentscited by 31
  5. 2025ThinkGuard: Deliberative Slow Thinking Leads to Cautious Guardrailscited by 29
  6. 2025ShieldGemma 2: Robust and Tractable Image Content Moderationcited by 27
  7. 2025Building a Foundational Guardrail for General Agentic Systems via Synthetic Datacited by 26
  8. 2024AdvPrefix: An Objective for Nuanced LLM Jailbreakscited by 25
  9. 2025The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMscited by 24
  10. 2025CultureGuard: Towards Culturally-Aware Dataset and Guard Model for Multilingual Safety Applicationscited by 24
  11. 2025RAG LLMs are Not Safer: A Safety Analysis of Retrieval-Augmented Generation for Large Language Modelscited by 23
  12. 2024On Calibration of LLM-based Guard Models for Reliable Content Moderationcited by 21
  13. 2025Steering Dialogue Dynamics for Robustness against Multi-turn Jailbreaking Attackscited by 20
  14. 2025BingoGuard: LLM Content Moderation Tools with Risk Levelscited by 20
  15. 2025FORTRESS: Frontier Risk Evaluation for National Security and Public Safetycited by 20
  16. 2024Endless Jailbreaks with Bijection Learningcited by 18
  17. 2025Enhancing LLM Safety Through a Theoretical Minimax Game Lenscited by 18
  18. 2025The Intrinsic Dimension of Prompts in Internal Representations of Large Language Modelscited by 17
  19. 2025SoK: Evaluating Jailbreak Guardrails for Large Language Modelscited by 17
  20. 2025DynaGuard: A Dynamic Guardian Model With User-Defined Policiescited by 16
  21. 2024Does Safety Training of LLMs Generalize to Semantically Related Natural Prompts?cited by 15
  22. 2025False Sense of Security: Why Probing-based Malicious Input Detection Fails to Generalizecited by 15
  23. 2024Attack Atlas: A Practitioner's Perspective on Challenges and Pitfalls in Red Teaming GenAIcited by 14
  24. 2025STACK: Adversarial Attacks on LLM Safeguard Pipelinescited by 14
  25. 2025Beyond Linear Probes: Dynamic Safety Monitoring for Language Modelscited by 14
  26. 2026YuFeng-XGuard: A Reasoning-Centric, Interpretable, and Flexible Guardrail Model for Large Language Modelscited by 14
  27. 2026SoK: The Attack Surface of Agentic AI - Tools and Autonomycited by 14
  28. 2024Beyond Interpretability: The Gains of Feature Monosemanticity on Model Robustnesscited by 13
  29. 2025Drift: Decoding-time Personalized Alignments with Implicit User Preferencescited by 13
  30. 2025Dialogue Injection Attack: Jailbreaking LLMs through Context Manipulationcited by 13
  31. 2025X-Guard: Multilingual Guard Agent for Content Moderationcited by 13
  32. 2025DC-AR: Efficient Masked Autoregressive Image Generation with Deep Compression Hybrid Tokenizercited by 13
  33. 2026CausalArmor: Efficient Indirect Prompt Injection Guardrails via Causal Attributioncited by 13
  34. 2026TraceSafe: A Systematic Assessment of LLM Guardrails on Multi-Step Tool-Calling Trajectoriescited by 12
  35. 2026Reflect-Guard: Enhancing LLM Safeguards against Adversarial Prompts via Logical Self-Reflectioncited by 12
  36. 2025AISafetyLab: A Comprehensive Framework for AI Safety Evaluation and Improvementcited by 11
  37. 2025Adversarial Tokenizationcited by 11
  38. 2024SafetyAnalyst: Interpretable, Transparent, and Steerable Safety Moderation for AI Behaviorcited by 10

50 most cited of 224 All 224 (JSON)