mlsanityData to 7 Oct 2026

Goodtriever: Adaptive Toxicity Mitigation with Retrieval-augmented Models

arXiv 2310.07589 ↗First posted 11 Oct 2023cs.AIcited by 23

On the map: Retrieval-augmented generation → Privacy and security of RAG

Abstract (arXiv)

Considerable effort has been dedicated to mitigating toxicity, but existing methods often require drastic modifications to model parameters or the use of computationally intensive auxiliary models. Furthermore, previous approaches have often neglected the crucial factor of language's evolving nature over time. In this work, we present a comprehensive perspective on toxicity mitigation that takes into account its changing nature. We introduce Goodtriever, a flexible methodology that matches the current state-of-the-art toxicity mitigation while achieving 43% relative latency reduction during inference and being more computationally efficient. By incorporating a retrieval-based approach at decoding time, Goodtriever enables toxicity-controlled text generation. Our research advocates for an increased focus on adaptable mitigation techniques, which better reflect the data drift models face when deployed in the wild. Code and data are available at https://github.com/for-ai/goodtriever.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation. Builds on 55 of its 74 references that are in the map. First posted on arXiv 11 Oct 2023.

Builds on

55 of 74 references.

References · 55

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  4. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  5. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  6. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  7. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  8. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  9. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  10. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,366
  11. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  12. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
38 more
  1. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  2. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  3. 2018Hierarchical Neural Story Generationcited by 1,182
  4. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,068
  5. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  6. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 793
  7. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
  8. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 754
  9. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 712
  10. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 711
  11. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 540
  12. 2021DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Expertscited by 434
  13. 2019Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classificationcited by 396
  14. 2020GeDi: Generative Discriminator Guided Sequence Generationcited by 377
  15. 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 356
  16. 2022Memorizing Transformerscited by 219
  17. 2020Nearest Neighbor Machine Translationcited by 209
  18. 2021Towards Continual Knowledge Learning of Language Modelscited by 172
  19. 2021Challenges in Detoxifying Language Modelscited by 169
  20. 2019LAMOL: LAnguage MOdeling for Lifelong Language Learningcited by 156
  21. 2022A Survey of Controllable Text Generation using Transformer-based Pre-trained Language Modelscited by 128
  22. 2021Lifelong Pretraining: Continually Adapting Language Models to Emerging Corporacited by 114
  23. 2022Training Language Models with Memory Augmentationcited by 112
  24. 2021Detoxifying Language Models Risks Marginalizing Minority Voicescited by 106
  25. 2019Augmenting Self-attention with Persistent Memorycited by 92
  26. 2021Adaptive Semiparametric Language Modelscited by 83
  27. 2022ELLE: Efficient Lifelong Pre-training for Emerging Datacited by 70
  28. 2022Exploring the Limits of Domain-Adaptive Training for Detoxifying Large-Scale Language Modelscited by 67
  29. 2022Neuro-Symbolic Language Modeling with Automaton-augmented Retrievalcited by 60
  30. 2021Text Detoxification using Large Pre-trained Neural Modelscited by 52
  31. 2023Intriguing Properties of Quantization at Scalecited by 48
  32. 2023On the Challenges of Using Black-Box APIs for Toxicity Evaluation in Researchcited by 44
  33. 2022Nonparametric Masked Language Modelingcited by 39
  34. 2021Fast Nearest Neighbor Machine Translationcited by 37
  35. 2022Detoxifying Text with MaRCo: Controllable Revision with Experts and Anti-Expertscited by 36
  36. 2022Is Neural Topic Modelling Better than Clustering? An Empirical Study on Clustering with Contextual Embeddings for Topicscited by 35
  37. 2022Unified Detoxifying and Debiasing in Language Generation via Inference-time Adaptive Optimizationcited by 32
  38. 2021Learning Kernel-Smoothed Machine Translation with Retrieved Examplescited by 25

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 15; 2025: 6; 2026: 2.

23 papers.

Sort:

By area:

  1. 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 292
  2. 2025MemOS: A Memory OS for AI Systemcited by 129
  3. 2024Enhancing Contextual Understanding in Large Language Models through Contrastive Decodingcited by 49
  4. 2024Representation Surgery: Theory and Practice of Affine Steeringcited by 47
  5. 2024Preference Tuning For Toxicity Mitigation Generalizes Across Languagescited by 31
  6. 2024From One to Many: Expanding the Scope of Toxicity Mitigation in Language Modelscited by 18
  7. 2024On the Limitations of Compute Thresholds as a Governance Strategycited by 17
  8. 2024Controlled Text Generation for Large Language Model with Dynamic Attribute Graphscited by 16
  9. 2024RSA-Control: A Pragmatics-Grounded Lightweight Controllable Text Generation Frameworkcited by 11
  10. 2024The Future of Open Human Feedbackcited by 10
  11. 2025The Multilingual Divide and Its Impact on Global AI Safetycited by 10
  12. 2024DESTEIN: Navigating Detoxification of Language Models via Universal Steering Pairs and Head-wise Activation Fusioncited by 9
11 more
  1. 2024The ShareLM Collection and Plugin: Contributing Human-Model Chats for the Benefit of the Communitycited by 8
  2. 2025UniDetox: Universal Detoxification of Large Language Models via Dataset Distillationcited by 8
  3. 2024Nexus: Specialization meets Adaptability for Efficiently Training Mixture of Expertscited by 7
  4. 2025IF-GUIDE: Influence Function-Guided Detoxification of LLMscited by 7
  5. 2024LIDAO: Towards Limited Interventions for Debiasing (Large) Language Modelscited by 3
  6. 2024Towards Robustness of Text-to-Visualization Translation against Lexical and Phrasal Variabilitycited by 2
  7. 2025Distributional Surgery for Language Model Activationscited by 2
  8. 2025Something Just Like TRuST : Toxicity Recognition of Span and Targetcited by 2
  9. 2024Fine-Grained Detoxification via Instance-Level Prefixes for Large Language Modelscited by 0
  10. 2026A Survey of Toxicity Detection and Mitigation Strategies for Multilingual Language Modelscited by 0
  11. 2026Beyond a Global Norm: Personalizing Toxicity Sensitivity in Language Models Without Retrainingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Text Detoxification: Data Efficiency, Semantic Preservation and Model Generalization
  2. 2026Where Does Toxicity Live? Mechanistic Localization and Targeted Suppression in Language Models
  3. 2025Rethinking Toxicity Evaluation in Large Language Models: A Multi-Label Perspective
  4. 2026Cleansing the Artificial Mind: A Self-Reflective Detoxification Framework for Large Language Models
  5. 2025Detoxifying Large Language Models via Autoregressive Reward Guided Representation Editing
  6. 2024Large Language Models can be Strong Self-Detoxifiers
  7. 2025<think> So let's replace this phrase with insult... </think> Lessons learned from generation of toxic texts with LLMs
  8. 2025ToxSearch: Evolving Prompts for Toxicity Search in Large Language Models