mlsanityData to 14 Oct 2026

A Data-Centric Approach for Safe and Secure Large Language Models against Threatening and Toxic Content

arXiv 2504.16120 ↗First posted 19 Apr 2025cs.CR cs.AIcited by 0

On the map: LLM safety and security → Safety alignment, refusal and guardrails

Abstract (arXiv)

Large Language Models (LLM) have made remarkable progress, but concerns about potential biases and harmful content persist. To address these apprehensions, we introduce a practical solution for ensuring LLM's safe and ethical use. Our novel approach focuses on a post-generation correction mechanism, the BART-Corrective Model, which adjusts generated content to ensure safety and security. Unlike relying solely on model fine-tuning or prompt engineering, our method provides a robust data-centric alternative for mitigating harmful content. We demonstrate the effectiveness of our approach through experiments on multiple toxic datasets, which show a significant reduction in mean toxicity and jail-breaking scores after integration. Specifically, our results show a reduction of 15% and 21% in mean toxicity and jail-breaking scores with GPT-4, a substantial reduction of 28% and 5% with PaLM2, a reduction of approximately 26% and 23% with Mistral-7B, and a reduction of 11.1% and 19% with Gemma-2b-it. These results demonstrate the potential of our approach to improve the safety and security of LLM, making them more suitable for real-world applications.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 23 of its 34 references that are in the map. First posted on arXiv 19 Apr 2025.

Builds on

23 of 34 references.

References · 23

  1. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  2. 2023Mistral 7Bcited by 3,164
  3. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
  4. 2024Gemma: Open Models Based on Gemini Research and Technologycited by 916
  5. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 783
  6. 2022Large Language Models Can Self-Improvecited by 664
  7. 2023Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervisioncited by 318
  8. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 238
  9. 2020HateBERT: Retraining BERT for Abusive Language Detection in Englishcited by 207
  10. 2023Self-correcting LLM-controlled Diffusion Modelscited by 113
  11. 2023Learning From Mistakes Makes LLM Better Reasonercited by 101
  12. 2024Confidence Matters: Revisiting Intrinsic Self-Correction Capabilities of Large Language Modelscited by 47
11 more
  1. 2022Toxicity Detection with Generative Prompt-based Inferencecited by 35
  2. 2024Model Editing as a Robust and Denoised variant of DPO: A Case Study on Toxicitycited by 35
  3. 2023Beyond Task Performance: Evaluating and Reducing the Flaws of Large Multimodal Models with In-Context Learningcited by 30
  4. 2023Interpretable Unified Language Checkingcited by 26
  5. 2024SELF-[IN]CORRECT: LLMs Struggle with Discriminating Self-Generated Responsescited by 26
  6. 2023Learning and Forgetting Unsafe Examples in Large Language Modelscited by 24
  7. 2023Efficient Toxic Content Detection by Bootstrapping and Distilling Large Language Modelscited by 21
  8. 2024AutoRAG-HP: Automatic Online Hyper-Parameter Tuning for Retrieval-Augmented Generationcited by 13
  9. 2023On the Intersection of Self-Correction and Trust in Language Modelscited by 12
  10. 2023Democratizing LLMs: An Exploration of Cost-Performance Trade-offs in Self-Refined Open-Source Modelscited by 11
  11. 2023N-Critics: Self-Refinement of Large Language Models with Ensemble of Criticscited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety
  2. 2025A Survey on Responsible LLMs: Inherent Risk, Malicious Use, and Mitigation Strategy
  3. 2025TRIDENT: Enhancing Large Language Model Safety with Tri-Dimensional Diversified Red-Teaming Data Synthesis
  4. 2024S-Eval: Towards Automated and Comprehensive Safety Evaluation for Large Language Models
  5. 2024Large Language Models are Vulnerable to Bait-and-Switch Attacks for Generating Harmful Content
  6. 2025Guardians and Offenders: A Survey on Harmful Content Generation and Safety Mitigation of LLM
  7. 2026Do Prompts Guarantee Safety? Mitigating Toxicity from LLM Generations through Subspace Intervention
  8. 2024Realistic Evaluation of Toxicity in Large Language Models