mlsanityData to 14 Oct 2026

Continually Improving Extractive QA via Human Feedback

arXiv 2305.12473 ↗First posted 21 May 2023cs.CL cs.AI cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

We study continually improving an extractive question answering (QA) system via human user feedback. We design and deploy an iterative approach, where information-seeking users ask questions, receive model-predicted answers, and provide feedback. We conduct experiments involving thousands of user interactions under diverse setups to broaden the understanding of learning from feedback over time. Our experiments show effective improvement from user feedback of extractive QA models over time across different data regimes, including significant potential for domain adaptation.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 35 of its 46 references that are in the map. First posted on arXiv 21 May 2023.

Builds on

35 of 46 references.

References · 35

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2017Deep reinforcement learning from human preferencescited by 3,912
  5. 2020Learning to summarize from human feedbackcited by 2,588
  6. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
  7. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  8. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  9. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  10. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  11. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  12. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,254
23 more
  1. 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
  2. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 488
  3. 2018QuAC : Question Answering in Contextcited by 381
  4. 2016Bidirectional Attention Flow for Machine Comprehensioncited by 278
  5. 2017Interactive Learning from Policy-Dependent Human Feedbackcited by 158
  6. 2023Training Language Models with Language Feedback at Scalecited by 128
  7. 2017Deep TAMER: Interactive Agent Shaping in High-Dimensional State Spacescited by 123
  8. 2017Learning a Neural Semantic Parser from User Feedbackcited by 99
  9. 2020Human-centric Dialog Training via Offline Reinforcement Learningcited by 93
  10. 2017Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedbackcited by 72
  11. 2018Can Neural Machine Translation be Improved with User Feedback?cited by 72
  12. 2018A Contextual Bandit Bake-offcited by 64
  13. 2018Reliability and Learnability of Human Bandit Feedback for Sequence-to-Sequence Reinforcement Learningcited by 57
  14. 2022Entity Cloze By Date: What LMs Know About Unseen Entitiescited by 55
  15. 2021Fool Me Twice: Entailment from Wikipedia Gamificationcited by 50
  16. 2016Learning Language Games through Interactioncited by 43
  17. 2022Learning New Skills after Deployment: Improving open-domain internet-driven dialogue with human feedbackcited by 36
  18. 2018Improving a Neural Semantic Parser by Counterfactual Learning from Human Bandit Feedbackcited by 32
  19. 2022Using Interactive Feedback to Improve the Accuracy and Explainability of Question Answering Systems Post-Deploymentcited by 25
  20. 2022Continual Learning for Instruction Following from Realtime Feedbackcited by 22
  21. 2021Continual Learning for Grounded Instruction Generation by Observing Human Following Behaviorcited by 19
  22. 2022Simulating Bandit Learning from User Feedback for Extractive Question Answeringcited by 14
  23. 2020Improving Conversational Question Answering Systems after Deployment using Feedback-Weighted Learningcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 7; 2025: 1; 2026: 1.

11 papers.

  1. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 238
  2. 2024RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMscited by 90
  3. 2024Position: Towards Bidirectional Human-AI Alignmentcited by 18
  4. 2025Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challengescited by 15
  5. 2023Investigating Table-to-Text Generation Capabilities of LLMs in Real-World Information Seeking Scenarioscited by 12
  6. 2024CoGen: Learning from Feedback with Coupled Comprehension and Generationcited by 7
  7. 2024I Could've Asked That: Reformulating Unanswerable Questionscited by 6
  8. 2024Retrospective Learning from Interactionscited by 6
  9. 2024DRS: Deep Question Reformulation With Structured Outputcited by 6
  10. 2024An Empirical Study on Self-correcting Large Language Models for Data Science Code Generationcited by 2
  11. 2026FactCorrector: A Graph-Inspired Approach to Long-Form Factuality Correction of Large Language Modelscited by 0