mlsanityData to 14 Oct 2026

WT5?! Training Text-to-Text Models to Explain their Predictions

arXiv 2004.14546 ↗First posted 30 Apr 2020cs.CL cs.LGcited by 148

Not on the map. Why?

Cited by 23 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Neural networks have recently achieved human-level performance on various challenging natural language processing (NLP) tasks, but it is notoriously difficult to understand why a neural network produced a particular prediction. In this paper, we leverage the text-to-text framework proposed by Raffel et al.(2019) to train language models to output a natural text explanation alongside their prediction. Crucially, this requires no modifications to the loss function or training and decoding procedures -- we simply train the model to output the explanation after generating the (natural text) prediction. We show that this approach not only obtains state-of-the-art results on explainability benchmarks, but also permits learning from a limited set of labeled explanations and transferring rationalization abilities across datasets. To facilitate reproducibility and future work, we release our code use to train the models.

Key facts. Cited by 148 papers in the map (data through 14 Oct 2026). First posted on arXiv 30 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 21; 2022: 35; 2023: 47; 2024: 27; 2025: 14; 2026: 4.

148 papers.

Sort:

By area:

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
  3. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 737
  4. 2022Large Language Models Can Self-Improvecited by 664
  5. 2022Teaching language models to support answers with verified quotescited by 292
  6. 2022Can language models learn from explanations in context?cited by 249
  7. 2022Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanationscited by 159
  8. 2022A Review on Language Models as Knowledge Basescited by 157
  9. 2023Symbolic Chain-of-Thought Distillation: Small Models Can Also "Think" Step-by-Stepcited by 148
  10. 2022Natural Language Descriptions of Deep Visual Featurescited by 137
  11. 2022Towards Faithful Model Explanation in NLP: A Surveycited by 130
  12. 2021Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processingcited by 126
38 more
  1. 2021Reframing Human-AI Collaboration for Generating Free-Text Explanationscited by 125
  2. 2022Explanations from Large Language Models Make Small Reasoners Bettercited by 124
  3. 2022Rationale-Augmented Ensembles in Language Modelscited by 111
  4. 2023Faithfulness Tests for Natural Language Explanationscited by 107
  5. 2021Do Transformer Modifications Transfer Across Implementations and Applications?cited by 87
  6. 2023Do Models Explain Themselves? Counterfactual Simulatability of Natural Language Explanationscited by 85
  7. 2021e-ViL: A Dataset and Benchmark for Natural Language Explanations in Vision-Language Taskscited by 84
  8. 2021Few-Shot Self-Rationalization with Natural Language Promptscited by 83
  9. 2022FLUTE: Figurative Language Understanding through Textual Explanationscited by 79
  10. 2021Identifying and Mitigating Spurious Correlations for Improving Robustness in NLP Modelscited by 76
  11. 2021When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Datacited by 75
  12. 2023On Measuring Faithfulness or Self-consistency of Natural Language Explanationscited by 73
  13. 2022Recitation-Augmented Language Modelscited by 65
  14. 2022FiD-Light: Efficient and Effective Retrieval-Augmented Text Generationcited by 63
  15. 2023Leveraging Large Language Models in Conversational Recommender Systemscited by 63
  16. 2023Explaining black box text modules in natural language with language modelscited by 62
  17. 2022NLX-GPT: A Model for Natural Language Explanations in Vision and Vision-Language Taskscited by 60
  18. 2021ExplaGraphs: An Explanation Graph Generation Task for Structured Commonsense Reasoningcited by 58
  19. 2022PINTO: Faithful Language Reasoning Using Prompt-Generated Rationalescited by 56
  20. 2023HAGRID: A Human-LLM Collaborative Dataset for Generative Information-Seeking with Attributioncited by 56
  21. 2023One Small Step for Generative AI, One Giant Leap for AGI: A Complete Survey on ChatGPT in AIGC Eracited by 55
  22. 2024Beyond Answers: Transferring Reasoning Capabilities to Smaller LLMs Using Multi-Teacher Knowledge Distillationcited by 47
  23. 2021Prompting Contrastive Explanations for Commonsense Reasoning Taskscited by 45
  24. 2023Are Machine Rationales (Not) Useful to Humans? Measuring and Improving Human Utility of Free-Text Rationalescited by 37
  25. 2022Can Rationalization Improve Robustness?cited by 36
  26. 2023Interpretable Long-Form Legal Question Answering with Retrieval-Augmented Large Language Modelscited by 35
  27. 2021UNIREX: A Unified Learning Framework for Language Model Rationale Extractioncited by 32
  28. 2021Local Interpretations for Explainable Natural Language Processing: A Surveycited by 31
  29. 2024How Interpretable are Reasoning Explanations from Prompting Large Language Models?cited by 29
  30. 2023Second Thoughts are Best: Learning to Re-Align With Human Values from Text Editscited by 27
  31. 2024On Behalf of the Stakeholders: Trends in NLP Model Interpretability in the Era of LLMscited by 25
  32. 2021Knowledge-Grounded Self-Rationalization via Extractive and Natural Language Explanationscited by 24
  33. 2024Properties and Challenges of LLM-Generated Explanationscited by 24
  34. 2022REV: Information-Theoretic Evaluation of Free-Text Rationalescited by 22
  35. 2023Rationalization for Explainable NLP: A Surveycited by 22
  36. 2023Leveraging Structured Information for Explainable Multi-hop Question Answering and Reasoningcited by 22
  37. 2023Passive learning of active causal strategies in agents and language modelscited by 21
  38. 2023Tailoring Self-Rationalizers with Multi-Reward Distillationcited by 21

50 most cited of 148 All 148 (JSON)