# Using Interactive Feedback to Improve the Accuracy and Explainability of Question Answering Systems Post-Deployment

arXiv: https://arxiv.org/abs/2204.03025 · HTML: https://mlsanity.com/paper/2204.03025/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 33 references that are in the map.
- First posted on arXiv 6 Apr 2022.

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [Reading Wikipedia to Answer Open-Domain Questions](https://mlsanity.com/paper/1704.00051/) (2017)
- [Latent Retrieval for Weakly Supervised Open Domain Question Answering](https://mlsanity.com/paper/1906.00300/) (2019)
- [CoQA: A Conversational Question Answering Challenge](https://mlsanity.com/paper/1808.07042/) (2018)
- [Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?](https://mlsanity.com/paper/2004.03685/) (2020)
- [e-SNLI: Natural Language Inference with Natural Language Explanations](https://mlsanity.com/paper/1812.01193/) (2018)
- [Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance](https://mlsanity.com/paper/2006.14779/) (2020)
- [Explain Yourself! Leveraging Language Models for Commonsense Reasoning](https://mlsanity.com/paper/1906.02361/) (2019)
- [End-to-End Open-Domain Question Answering with BERTserini](https://mlsanity.com/paper/1902.01718/) (2019)
- [ParlAI: A Dialog Research Software Platform](https://mlsanity.com/paper/1705.06476/) (2017)
- [Learning from Dialogue after Deployment: Feed Yourself, Chatbot!](https://mlsanity.com/paper/1901.05415/) (2019)
- [NILE : Natural Language Inference with Faithful Natural Language Explanations](https://mlsanity.com/paper/2005.12116/) (2020)
- [QED: A Framework and Dataset for Explanations in Question Answering](https://mlsanity.com/paper/2009.06354/) (2020)
- [Dialogue Learning With Human-In-The-Loop](https://mlsanity.com/paper/1611.09823/) (2016)
- [ExpBERT: Representation Engineering with Natural Language Explanations](https://mlsanity.com/paper/2005.01932/) (2020)
- [Dialog-based Language Learning](https://mlsanity.com/paper/1604.06045/) (2016)
- [Learning from Explanations with Neural Execution Tree](https://mlsanity.com/paper/1911.01352/) (2019)
- [Adaptive Document Retrieval for Deep Question Answering](https://mlsanity.com/paper/1808.06528/) (2018)
- [Improving Conversational Question Answering Systems after Deployment using Feedback-Weighted Learning](https://mlsanity.com/paper/2011.00615/) (2020)

## Built on by (most cited first, 25 of 25)

- [Training Language Models with Language Feedback at Scale](https://mlsanity.com/paper/2303.16755/) (2023)
- [Are self-explanations from Large Language Models faithful?](https://mlsanity.com/paper/2401.07927/) (2024)
- [RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs](https://mlsanity.com/paper/2404.08555/) (2024)
- [Foundational Autoraters: Taming Large Language Models for Better Automatic Evaluation](https://mlsanity.com/paper/2407.10817/) (2024)
- [Aligning LLM Agents by Learning Latent Preference from User Edits](https://mlsanity.com/paper/2404.15269/) (2024)
- [Bridging the Gap: A Survey on Integrating (Human) Feedback for Natural Language Generation](https://mlsanity.com/paper/2305.00955/) (2023)
- [Training Language Models with Language Feedback](https://mlsanity.com/paper/2204.14146/) (2022)
- [Learning to Simulate Natural Language Feedback for Interactive Semantic Parsing](https://mlsanity.com/paper/2305.08195/) (2023)
- [Position: Towards Bidirectional Human-AI Alignment](https://mlsanity.com/paper/2406.09264/) (2024)
- [Black-box Uncertainty Quantification Method for LLM-as-a-Judge](https://mlsanity.com/paper/2410.11594/) (2024)
- [Reasons to Reject? Aligning Language Models with Judgments](https://mlsanity.com/paper/2312.14591/) (2023)
- [Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challenges](https://mlsanity.com/paper/2507.19672/) (2025)
- [CARE: Confounder-Aware Aggregation for Reliable LLM Evaluation](https://mlsanity.com/paper/2603.00039/) (2026)
- [Continually Improving Extractive QA via Human Feedback](https://mlsanity.com/paper/2305.12473/) (2023)
- [Robust Question Answering against Distribution Shifts with Test-Time Adaptation: An Empirical Study](https://mlsanity.com/paper/2302.04618/) (2023)
- [Honesty to Subterfuge: In-Context Reinforcement Learning Can Make Honest Models Reward Hack](https://mlsanity.com/paper/2410.06491/) (2024)
- [InterFair: Debiasing with Natural Language Feedback for Fair Interpretable Predictions](https://mlsanity.com/paper/2210.07440/) (2022)
- [Reranking Laws for Language Generation: A Communication-Theoretic Perspective](https://mlsanity.com/paper/2409.07131/) (2024)
- [What if you said that differently?: How Explanation Formats Affect Human Feedback Efficacy and User Perception](https://mlsanity.com/paper/2311.09558/) (2023)
- [Aligning Black-box Language Models with Human Judgments](https://mlsanity.com/paper/2502.04997/) (2025)
- [MWE as WSD: Solving Multiword Expression Identification with Word Sense Disambiguation](https://mlsanity.com/paper/2303.06623/) (2023)
- [Multilingual Datasets for Custom Input Extraction and Explanation Requests Parsing in Conversational XAI Systems](https://mlsanity.com/paper/2508.14982/) (2025)
- [Modular Expert Merging for Biomedical Retrieval](https://mlsanity.com/paper/2602.04731/) (2026)
- [H+ Embedding: Harmonizing Global and Token-Level Retrieval with Context-Dependent Phrases](https://mlsanity.com/paper/2608.00065/) (2026)
- [Evidence Lock Before Commitment: A Frozen Interface Degrades LLM-as-Judge Evaluation](https://mlsanity.com/paper/2608.05353/) (2026)

## Abstract (arXiv)

Most research on question answering focuses on the pre-deployment stage; i.e., building an accurate model for deployment. In this paper, we ask the question: Can we improve QA systems further post-deployment based on user interactions? We focus on two kinds of improvements: 1) improving the QA system's performance itself, and 2) providing the model with the ability to explain the correctness or incorrectness of an answer. We collect a retrieval-based QA dataset, FeedbackQA, which contains interactive feedback from users. We collect this dataset by deploying a base QA system to crowdworkers who then engage with the system and provide feedback on the quality of its answers. The feedback contains both structured ratings and unstructured natural language explanations. We train a neural model with this feedback data that can generate explanations and re-score answer candidates. We show that feedback data not only improves the accuracy of the deployed QA system but also other stronger non-deployed systems. The generated explanations also help users make informed decisions about the correctness of answers. Project page: https://mcgill-nlp.github.io/feedbackqa/