# Elastic Weight Removal for Faithful and Abstractive Dialogue Generation

arXiv: https://arxiv.org/abs/2303.17574 · HTML: https://mlsanity.com/paper/2303.17574/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 56 of its 80 references that are in the map.
- First posted on arXiv 30 Mar 2023.

## Builds on (most cited first, 30 of 56)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [LaMDA: Language Models for Dialog Applications](https://mlsanity.com/paper/2201.08239/) (2022)
- [Editing Models with Task Arithmetic](https://mlsanity.com/paper/2212.04089/) (2022)
- [A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity](https://mlsanity.com/paper/2302.04023/) (2023)
- [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://mlsanity.com/paper/1909.05858/) (2019)
- [The Flan Collection: Designing Data and Methods for Effective Instruction Tuning](https://mlsanity.com/paper/2301.13688/) (2023)
- [Retrieval Augmentation Reduces Hallucination in Conversation](https://mlsanity.com/paper/2104.07567/) (2021)
- [Merging Models with Fisher-Weighted Averaging](https://mlsanity.com/paper/2111.09832/) (2021)
- [Recipes for building an open-domain chatbot](https://mlsanity.com/paper/2004.13637/) (2020)
- [Wizard of Wikipedia: Knowledge-Powered Conversational agents](https://mlsanity.com/paper/1811.01241/) (2018)
- [Is ChatGPT a General-Purpose Natural Language Processing Task Solver?](https://mlsanity.com/paper/2302.06476/) (2023)
- [Git Re-Basin: Merging Models modulo Permutation Symmetries](https://mlsanity.com/paper/2209.04836/) (2022)
- [New insights and perspectives on the natural gradient method](https://mlsanity.com/paper/1412.1193/) (2014)
- [DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts](https://mlsanity.com/paper/2105.03023/) (2021)
- [Parameter-Efficient Transfer Learning with Diff Pruning](https://mlsanity.com/paper/2012.07463/) (2020)
- [Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategies](https://mlsanity.com/paper/1804.11283/) (2018)
- [FEQA: A Question Answering Evaluation Framework for Faithfulness Assessment in Abstractive Summarization](https://mlsanity.com/paper/2005.03754/) (2020)
- [ToTTo: A Controlled Table-To-Text Generation Dataset](https://mlsanity.com/paper/2004.14373/) (2020)
- [Measuring Attribution in Natural Language Generation Models](https://mlsanity.com/paper/2112.12870/) (2021)
- [Quark: Controllable Text Generation with Reinforced Unlearning](https://mlsanity.com/paper/2205.13636/) (2022)
- [Challenges in Data-to-Document Generation](https://mlsanity.com/paper/1707.08052/) (2017)
- [Limitations of the Empirical Fisher Approximation for Natural Gradient Descent](https://mlsanity.com/paper/1905.12558/) (2019)
- [Training Neural Networks with Fixed Sparse Masks](https://mlsanity.com/paper/2111.09839/) (2021)
- [The Curious Case of Hallucinations in Neural Machine Translation](https://mlsanity.com/paper/2104.06683/) (2021)

## Built on by (most cited first, 29 of 29)

- [Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewards](https://mlsanity.com/paper/2306.04488/) (2023)
- [WARM: On the Benefits of Weight Averaged Reward Models](https://mlsanity.com/paper/2401.12187/) (2024)
- [MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problems](https://mlsanity.com/paper/2305.14536/) (2023)
- [Deep Model Fusion: A Survey](https://mlsanity.com/paper/2309.15698/) (2023)
- [Cognitive Mirage: A Review of Hallucinations in Large Language Models](https://mlsanity.com/paper/2309.06794/) (2023)
- [Model Merging by Uncertainty-Based Gradient Matching](https://mlsanity.com/paper/2310.12808/) (2023)
- [Chat Vector: A Simple Approach to Equip LLMs with Instruction Following and Model Alignment in New Languages](https://mlsanity.com/paper/2310.04799/) (2023)
- [Calibrating Verbal Uncertainty as a Linear Feature to Reduce Hallucinations](https://mlsanity.com/paper/2503.14477/) (2025)
- [Detecting and Mitigating Hallucinations in Multilingual Summarisation](https://mlsanity.com/paper/2305.13632/) (2023)
- [Faithful Persona-based Conversational Dataset Generation with Large Language Models](https://mlsanity.com/paper/2312.10007/) (2023)
- [UnIVAL: Unified Model for Image, Video, Audio and Language Tasks](https://mlsanity.com/paper/2307.16184/) (2023)
- [Stepwise Verification and Remediation of Student Reasoning Errors with Large Language Model Tutors](https://mlsanity.com/paper/2407.09136/) (2024)
- [Digital Forgetting in Large Language Models: A Survey of Unlearning Methods](https://mlsanity.com/paper/2404.02062/) (2024)
- [Book2Dial: Generating Teacher-Student Interactions from Textbooks for Cost-Effective Development of Educational Chatbots](https://mlsanity.com/paper/2403.03307/) (2024)
- [Mask-DPO: Generalizable Fine-grained Factuality Alignment of LLMs](https://mlsanity.com/paper/2503.02846/) (2025)
- [Configurable Foundation Models: Building LLMs from a Modular Perspective](https://mlsanity.com/paper/2409.02877/) (2024)
- [Task Arithmetic can Mitigate Synthetic-to-Real Gap in Automatic Speech Recognition](https://mlsanity.com/paper/2406.02925/) (2024)
- [ANAH-v2: Scaling Analytical Hallucination Annotation of Large Language Models](https://mlsanity.com/paper/2407.04693/) (2024)
- [Realistic Evaluation of Model Merging for Compositional Generalization](https://mlsanity.com/paper/2409.18314/) (2024)
- [textitDial BeInfo for Faithfulness: Improving Factuality of Information-Seeking Dialogue via Behavioural Fine-Tuning](https://mlsanity.com/paper/2311.09800/) (2023)
- [ANAH: Analytical Annotation of Hallucinations in Large Language Models](https://mlsanity.com/paper/2405.20315/) (2024)
- [Think While You Write: Hypothesis Verification Promotes Faithful Knowledge-to-Text Generation](https://mlsanity.com/paper/2311.09467/) (2023)
- [Investigating Task Arithmetic for Zero-Shot Information Retrieval](https://mlsanity.com/paper/2505.00649/) (2025)
- [Red Teaming Language Models for Processing Contradictory Dialogues](https://mlsanity.com/paper/2405.10128/) (2024)
- [Decision-Making with Deliberation: Meta-reviewing as a Document-grounded Dialogue](https://mlsanity.com/paper/2508.05283/) (2025)
- [Investigating Content Planning for Navigating Trade-offs in Knowledge-Grounded Dialogue](https://mlsanity.com/paper/2402.02077/) (2024)
- [Task Vector Bases: A Unified and Scalable Framework for Compressed Task Arithmetic](https://mlsanity.com/paper/2502.01015/) (2025)
- [DKL: Decoupled Knowledge Learning for Instruction-Tuned Language Models](https://mlsanity.com/paper/2609.02685/) (2026)
- [CHARP: Conversation History AwaReness Probing for Knowledge-grounded Dialogue Systems](https://mlsanity.com/paper/2405.15110/) (2024)

## Abstract (arXiv)

Ideally, dialogue systems should generate responses that are faithful to the knowledge contained in relevant documents. However, many models generate hallucinated responses instead that contradict it or contain unverifiable information. To mitigate such undesirable behaviour, it has been proposed to fine-tune a `negative expert' on negative examples and subtract its parameters from those of a pre-trained model. However, intuitively, this does not take into account that some parameters are more responsible than others in causing hallucinations. Thus, we propose to weigh their individual importance via (an approximation of) the Fisher Information matrix, which measures the uncertainty of their estimate. We call this method Elastic Weight Removal (EWR). We evaluate our method -- using different variants of Flan-T5 as a backbone language model -- on multiple datasets for information-seeking dialogue generation and compare our method with state-of-the-art techniques for faithfulness, such as CTRL, Quark, DExperts, and Noisy Channel reranking. Extensive automatic and human evaluation shows that EWR systematically increases faithfulness at minor costs in terms of other metrics. However, we notice that only discouraging hallucinations may increase extractiveness, i.e. shallow copy-pasting of document spans, which can be undesirable. Hence, as a second main contribution, we show that our method can be extended to simultaneously discourage hallucinations and extractive responses. We publicly release the code for reproducing EWR and all baselines.