# Explaining Question Answering Models through Text Generation

arXiv: https://arxiv.org/abs/2004.05569 · HTML: https://mlsanity.com/paper/2004.05569/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Apr 2020.

## Built on by (most cited first, 30 of 33)

- [Teaching language models to support answers with verified quotes](https://mlsanity.com/paper/2203.11147/) (2022)
- [Generated Knowledge Prompting for Commonsense Reasoning](https://mlsanity.com/paper/2110.08387/) (2021)
- [Post-hoc Interpretability for Neural NLP: A Survey](https://mlsanity.com/paper/2108.04840/) (2021)
- [Reframing Human-AI Collaboration for Generating Free-Text Explanations](https://mlsanity.com/paper/2112.08674/) (2021)
- [Are self-explanations from Large Language Models faithful?](https://mlsanity.com/paper/2401.07927/) (2024)
- [Few-Shot Self-Rationalization with Natural Language Prompts](https://mlsanity.com/paper/2111.08284/) (2021)
- [Prompting Contrastive Explanations for Commonsense Reasoning Tasks](https://mlsanity.com/paper/2106.06823/) (2021)
- [Grounding 'Grounding' in NLP](https://mlsanity.com/paper/2106.02192/) (2021)
- [Rainier: Reinforced Knowledge Introspector for Commonsense Question Answering](https://mlsanity.com/paper/2210.03078/) (2022)
- [Interpretable Long-Form Legal Question Answering with Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2309.17050/) (2023)
- [Flexible Generation of Natural Language Deductions](https://mlsanity.com/paper/2104.08825/) (2021)
- [Generic Temporal Reasoning with Differential Analysis and Explanation](https://mlsanity.com/paper/2212.10467/) (2022)
- [METGEN: A Module-Based Entailment Tree Generation Framework for Answer Explanation](https://mlsanity.com/paper/2205.02593/) (2022)
- [Few-Shot Out-of-Domain Transfer Learning of Natural Language Explanations in a Label-Abundant Setup](https://mlsanity.com/paper/2112.06204/) (2021)
- [Crystal: Introspective Reasoners Reinforced with Self-Feedback](https://mlsanity.com/paper/2310.04921/) (2023)
- [Elevating Legal LLM Responses: Harnessing Trainable Logical Structures and Semantic Knowledge with Legal Reasoning](https://mlsanity.com/paper/2502.07912/) (2025)
- [Explicit Planning Helps Language Models in Logical Reasoning](https://mlsanity.com/paper/2303.15714/) (2023)
- [Unifying Model Explainability and Robustness for Joint Text Classification and Rationale Extraction](https://mlsanity.com/paper/2112.10424/) (2021)
- [Summarize-then-Answer: Generating Concise Explanations for Multi-hop Reading Comprehension](https://mlsanity.com/paper/2109.06853/) (2021)
- [Beyond Labels: Empowering Human Annotators with Natural Language Explanations through a Novel Active-Learning Architecture](https://mlsanity.com/paper/2305.12710/) (2023)
- [Clues Before Answers: Generation-Enhanced Multiple-Choice QA](https://mlsanity.com/paper/2205.00274/) (2022)
- [Elaboration-Generating Commonsense Question Answering at Scale](https://mlsanity.com/paper/2209.01232/) (2022)
- [Generating Full Length Wikipedia Biographies: The Impact of Gender Bias on the Retrieval-Based Generation of Women Biographies](https://mlsanity.com/paper/2204.05879/) (2022)
- [Interactive Question Answering Systems: Literature Review](https://mlsanity.com/paper/2209.01621/) (2022)
- [Enhancing Reasoning Abilities of Small LLMs with Cognitive Alignment](https://mlsanity.com/paper/2504.09802/) (2025)
- [Are Training Resources Insufficient? Predict First Then Explain!](https://mlsanity.com/paper/2110.02056/) (2021)
- [HPE:Answering Complex Questions over Text by Hybrid Question Parsing and Execution](https://mlsanity.com/paper/2305.07789/) (2023)
- [From Complex to Simple: Unraveling the Cognitive Tree for Reasoning with Small Language Models](https://mlsanity.com/paper/2311.06754/) (2023)
- [Explanation Regeneration via Information Bottleneck](https://mlsanity.com/paper/2212.09603/) (2022)
- [Generating Hypothetical Events for Abductive Inference](https://mlsanity.com/paper/2106.03973/) (2021)

## Abstract (arXiv)

Large pre-trained language models (LMs) have been shown to perform surprisingly well when fine-tuned on tasks that require commonsense and world knowledge. However, in end-to-end architectures, it is difficult to explain what is the knowledge in the LM that allows it to make a correct prediction. In this work, we propose a model for multi-choice question answering, where a LM-based generator generates a textual hypothesis that is later used by a classifier to answer the question. The hypothesis provides a window into the information used by the fine-tuned LM that can be inspected by humans. A key challenge in this setup is how to constrain the model to generate hypotheses that are meaningful to humans. We tackle this by (a) joint training with a simple similarity classifier that encourages meaningful hypotheses, and (b) by adding loss functions that encourage natural text without repetitions. We show on several tasks that our model reaches performance that is comparable to end-to-end architectures, while producing hypotheses that elucidate the knowledge used by the LM for answering the question.