# ReSeTOX: Re-learning attention weights for toxicity mitigation in machine translation

arXiv: https://arxiv.org/abs/2305.11761 · HTML: https://mlsanity.com/paper/2305.11761/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 22 references that are in the map.
- First posted on arXiv 19 May 2023.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [Plug and Play Language Models: A Simple Approach to Controlled Text Generation](https://mlsanity.com/paper/1912.02164/) (2019)
- [Language-agnostic BERT Sentence Embedding](https://mlsanity.com/paper/2007.01852/) (2020)
- [A Holistic Approach to Undesired Content Detection in the Real World](https://mlsanity.com/paper/2208.03274/) (2022)
- [The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics](https://mlsanity.com/paper/2102.01672/) (2021)
- [Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasets](https://mlsanity.com/paper/2106.10328/) (2021)
- [Recipes for Safety in Open-domain Chatbots](https://mlsanity.com/paper/2010.07079/) (2020)
- ["I'm sorry to hear that": Finding New Biases in Language Models with a Holistic Descriptor Dataset](https://mlsanity.com/paper/2205.09209/) (2022)
- [ZeroCap: Zero-Shot Image-to-Text Generation for Visual-Semantic Arithmetic](https://mlsanity.com/paper/2111.14447/) (2021)
- [Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings Alone Do Well, Sentence Similarity Even Better](https://mlsanity.com/paper/2212.08597/) (2022)
- [Sampling-Based Approximations to Minimum Bayes Risk Decoding for Neural Machine Translation](https://mlsanity.com/paper/2108.04718/) (2021)
- [Towards Opening the Black Box of Neural Machine Translation: Source and Target Interpretations of the Transformer](https://mlsanity.com/paper/2205.11631/) (2022)
- [Toxicity in Multilingual Machine Translation at Scale](https://mlsanity.com/paper/2210.03070/) (2022)
- [Zero-Shot Video Captioning with Evolving Pseudo-Tokens](https://mlsanity.com/paper/2207.11100/) (2022)
- [Reward Modeling for Mitigating Toxicity in Transformer-based Language Models](https://mlsanity.com/paper/2202.09662/) (2022)

## Built on by (most cited first, 3 of 3)

- [Added Toxicity Mitigation at Inference Time for Multimodal and Massively Multilingual Translation](https://mlsanity.com/paper/2311.06532/) (2023)
- [Redefining Experts: Interpretable Decomposition of Language Models for Toxicity Mitigation](https://mlsanity.com/paper/2509.16660/) (2025)
- [MT-LENS: An all-in-one Toolkit for Better Machine Translation Evaluation](https://mlsanity.com/paper/2412.11615/) (2024)

## Abstract (arXiv)

Our proposed method, ReSeTOX (REdo SEarch if TOXic), addresses the issue of Neural Machine Translation (NMT) generating translation outputs that contain toxic words not present in the input. The objective is to mitigate the introduction of toxic language without the need for re-training. In the case of identified added toxicity during the inference process, ReSeTOX dynamically adjusts the key-value self-attention weights and re-evaluates the beam search hypotheses. Experimental results demonstrate that ReSeTOX achieves a remarkable 57% reduction in added toxicity while maintaining an average translation quality of 99.5% across 164 languages.