# FiD-Ex: Improving Sequence-to-Sequence Models for Extractive Rationale Generation

arXiv: https://arxiv.org/abs/2012.15482 · HTML: https://mlsanity.com/paper/2012.15482/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Dec 2020.

## Built on by (most cited first, 21 of 21)

- [FiD-Light: Efficient and Effective Retrieval-Augmented Text Generation](https://mlsanity.com/paper/2209.14290/) (2022)
- [Why So Gullible? Enhancing the Robustness of Retrieval-Augmented Models against Counterfactual Noise](https://mlsanity.com/paper/2305.01579/) (2023)
- [Interpretable Long-Form Legal Question Answering with Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2309.17050/) (2023)
- [UNIREX: A Unified Learning Framework for Language Model Rationale Extraction](https://mlsanity.com/paper/2112.08802/) (2021)
- [Rationalization for Explainable NLP: A Survey](https://mlsanity.com/paper/2301.08912/) (2023)
- [Investigating the Benefits of Free-Form Rationales](https://mlsanity.com/paper/2206.11083/) (2022)
- [Explainability in Practice: A Survey of Explainable NLP Across Various Domains](https://mlsanity.com/paper/2502.00837/) (2025)
- [FRAME: Evaluating Rationale-Label Consistency Metrics for Free-Text Rationales](https://mlsanity.com/paper/2207.00779/) (2022)
- [Towards Faithful Explanations: Boosting Rationalization with Shortcuts Discovery](https://mlsanity.com/paper/2403.07955/) (2024)
- [Generating Full Length Wikipedia Biographies: The Impact of Gender Bias on the Retrieval-Based Generation of Women Biographies](https://mlsanity.com/paper/2204.05879/) (2022)
- [Plausible Extractive Rationalization through Semi-Supervised Entailment Signal](https://mlsanity.com/paper/2402.08479/) (2024)
- [Exploring the Trade-off Between Model Performance and Explanation Plausibility of Text Classifiers Using Human Rationales](https://mlsanity.com/paper/2404.03098/) (2024)
- [REFER: An End-to-end Rationale Extraction Framework for Explanation Regularization](https://mlsanity.com/paper/2310.14418/) (2023)
- [ClaimDiff: Comparing and Contrasting Claims on Contentious Issues](https://mlsanity.com/paper/2205.12221/) (2022)
- [Towards Reconciling Usability and Usefulness of Explainable AI Methodologies](https://mlsanity.com/paper/2301.05347/) (2023)
- [Optimizing Factual Accuracy in Text Generation through Dynamic Knowledge Selection](https://mlsanity.com/paper/2308.15711/) (2023)
- [Multi-Granularity Guided Fusion-in-Decoder](https://mlsanity.com/paper/2404.02581/) (2024)
- [Learning to Generate and Evaluate Fact-checking Explanations with Transformers](https://mlsanity.com/paper/2410.15669/) (2024)
- [Make Your Decision Convincing! A Unified Two-Stage Framework: Self-Attribution and Decision-Making](https://mlsanity.com/paper/2310.13610/) (2023)
- [Corrections Meet Explanations: A Unified Framework for Explainable Grammatical Error Correction](https://mlsanity.com/paper/2502.15261/) (2025)
- [Structures Facilitate Retrieve, Rerank, and Generate](https://mlsanity.com/paper/2606.03247/) (2026)

## Abstract (arXiv)

Natural language (NL) explanations of model predictions are gaining popularity as a means to understand and verify decisions made by large black-box pre-trained models, for NLP tasks such as Question Answering (QA) and Fact Verification. Recently, pre-trained sequence to sequence (seq2seq) models have proven to be very effective in jointly making predictions, as well as generating NL explanations. However, these models have many shortcomings; they can fabricate explanations even for incorrect predictions, they are difficult to adapt to long input documents, and their training requires a large amount of labeled data. In this paper, we develop FiD-Ex, which addresses these shortcomings for seq2seq models by: 1) introducing sentence markers to eliminate explanation fabrication by encouraging extractive generation, 2) using the fusion-in-decoder architecture to handle long input contexts, and 3) intermediate fine-tuning on re-structured open domain QA datasets to improve few-shot performance. FiD-Ex significantly improves over prior work in terms of explanation metrics and task accuracy, on multiple tasks from the ERASER explainability benchmark, both in the fully supervised and in the few-shot settings.