mlsanityData to 7 Oct 2026

Learning to Repair: Repairing model output errors after deployment using a dynamic memory of feedback

arXiv 2112.09737 ↗First posted 16 Dec 2021cs.CL cs.AIcited by 35

Not on the map. Why?

Abstract (arXiv)

Large language models (LMs), while powerful, are not immune to mistakes, but can be difficult to retrain. Our goal is for an LM to continue to improve after deployment, without retraining, using feedback from the user. Our approach pairs an LM with (i) a growing memory of cases where the user identified an output error and provided general feedback on how to correct it (ii) a corrector model, trained to translate this general feedback into specific edits to repair the model output. Given a new, unseen input, our model can then use feedback from similar, past cases to repair output errors that may occur. We instantiate our approach using an existing, fixed model for script generation, that takes a goal (e.g., "bake a cake") and generates a partially ordered sequence of actions to achieve that goal, sometimes containing errors. Our memory-enhanced system, FBNet, learns to apply user feedback to repair such errors (up to 30 points improvement), while making a start at avoiding similar past mistakes on new, unseen examples (up to 7 points improvement in a controlled setting). This is a first step towards strengthening deployed models, potentially broadening their utility. Our code and data is available at https://github.com/allenai/interscript/.

Key facts. Cited by 35 papers in the map (data through 7 Oct 2026). Builds on 16 of its 36 references that are in the map. First posted on arXiv 16 Dec 2021.

Builds on

16 of 36 references.

References · 16

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  5. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  6. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  7. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  8. 2021The GEM Benchmark: Natural Language Generation, its Evaluation and Metricscited by 239
  9. 2020Graph-based, Self-Supervised Program Repair from Diagnostic Feedbackcited by 93
  10. 2021Break-It-Fix-It: Unsupervised Learning for Program Repaircited by 71
  11. 2021BeliefBank: Adding Memory to a Pre-Trained Language Model for a Systematic Notion of Beliefcited by 56
  12. 2016Learning Language Games through Interactioncited by 43
4 more
  1. 2021NL-EDIT: Correcting semantic parse errors through natural language interactioncited by 33
  2. 2020Leap-Of-Thought: Teaching Pre-Trained Models to Systematically Reason Over Implicit Knowledgecited by 24
  3. 2021proScript: Partially Ordered Scripts Generation via Pre-trained Language Modelscited by 21
  4. 2019Improving Natural Language Interaction with Robots Using Advicecited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 17; 2024: 7; 2025: 6; 2026: 1.

35 papers.

Sort:

By area:

  1. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  2. 2023REFINER: Reasoning Feedback on Intermediate Representationscited by 203
  3. 2023RL4F: Generating Natural Language Feedback with Reinforcement Learning for Repairing Model Outputscited by 104
  4. 2023Bridging the Gap: A Survey on Integrating (Human) Feedback for Natural Language Generationcited by 55
  5. 2023LLM-based Medical Assistant Personalization with Short- and Long-Term Memory Coordinationcited by 46
  6. 2022Towards Teachable Reasoning Systems: Using a Dynamic Memory of User Feedback for Continual System Improvementcited by 37
  7. 2025Rethinking Memory in LLM based Agents: Representations, Operations, and Emerging Topicscited by 36
  8. 2023How Do Large Language Models Capture the Ever-changing World Knowledge? A Review of Recent Advancescited by 33
  9. 2024Personalized LLM Response Generation with Parameterized Memory Injectioncited by 33
  10. 2023Improving Grounded Language Understanding in a Collaborative Environment by Interacting with Agents Through Help Feedbackcited by 31
  11. 2023CHATREPORT: Democratizing Sustainability Disclosure Analysis through LLM-based Toolscited by 29
  12. 2023Editing Common Sense in Transformerscited by 28
23 more
  1. 2024Large Model Based Agents: State-of-the-Art, Cooperation Paradigms, Security and Privacy, and Future Trendscited by 26
  2. 2022When Life Gives You Lemons, Make Cherryade: Converting Feedback from Bad Responses into Good Labelscited by 20
  3. 2023Learning by Self-Explainingcited by 17
  4. 2023Reasons to Reject? Aligning Language Models with Judgmentscited by 15
  5. 2024THOUGHTSCULPT: Reasoning with Intermediate Revision and Searchcited by 11
  6. 2024Could ChatGPT get an Engineering Degree? Evaluating Higher Education Vulnerability to AI Assistantscited by 10
  7. 2023Let Me Teach You: Pedagogical Foundations of Feedback for Language Modelscited by 9
  8. 2023Constructive Large Language Models Alignment with Diverse Feedbackcited by 9
  9. 2024Preemptive Detection and Correction of Misaligned Actions in LLM Agentscited by 9
  10. 2025Bring Your Own Knowledge: A Survey of Methods for LLM Knowledge Expansioncited by 8
  11. 2022InterFair: Debiasing with Natural Language Feedback for Fair Interpretable Predictionscited by 7
  12. 2025Inverse Reinforcement Learning Meets Large Language Model Post-Training: Basics, Advances, and Opportunitiescited by 7
  13. 2024RAM: Towards an Ever-Improving Memory System by Learning from Communicationscited by 6
  14. 2023Interpreting User Requests in the Context of Natural Language Standing Instructionscited by 5
  15. 2023System-Level Natural Language Feedbackcited by 4
  16. 2023LUNA: A Model-Based Universal Analysis Framework for Large Language Modelscited by 4
  17. 2025How to Enable Effective Cooperation Between Humans and NLP Models: A Survey of Principles, Formalizations, and Beyondcited by 4
  18. 2023Interactively Learning Social Media Representations Improves News Source Factuality Detectioncited by 3
  19. 2025Recursive Decomposition of Logical Thoughts: Framework for Superior Reasoning and Knowledge Propagation in Large Language Modelscited by 3
  20. 2022Memory-assisted prompt editing to improve GPT-3 after deploymentcited by 2
  21. 2024A Zero-Shot Open-Vocabulary Pipeline for Dialogue Understandingcited by 2
  22. 2025CLEAR: Contrasting Textual Feedback with Experts and Amateurs for Reasoningcited by 0
  23. 2026Evaluating Agentic Learning Harness Capabilities Without Labels via the Scaling Hypothesiscited by 0