mlsanityData to 7 Oct 2026

Improving Neural Model Performance through Natural Language Feedback on Their Explanations

arXiv 2104.08765 ↗First posted 18 Apr 2021cs.CLcited by 5

On the map: LLM alignment / RLHF / preference optimisation → Process reward models for reasoning

Abstract (arXiv)

A class of explainable NLP models for reasoning tasks support their decisions by generating free-form or structured explanations, but what happens when these supporting structures contain errors? Our goal is to allow users to interactively correct explanation structures through natural language feedback. We introduce MERCURIE - an interactive system that refines its explanations for a given reasoning task by getting human feedback in natural language. Our approach generates graphs that have 40% fewer inconsistencies as compared with the off-the-shelf system. Further, simply appending the corrected explanation structures to the output leads to a gain of 1.2 points on accuracy on defeasible reasoning across all three domains. We release a dataset of over 450k graphs for defeasible reasoning generated by our system at https://tinyurl.com/mercurie .

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Process reward models for reasoning, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 12 of its 25 references that are in the map. First posted on arXiv 18 Apr 2021.

Builds on

12 of 25 references.

References · 12

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  4. 2019WIQA: A dataset for "What if..." reasoning over procedural textcited by 71
  5. 2020Speak to your Parser: Interactive Text-to-SQL with Natural Language Feedbackcited by 44
  6. 2016Learning Language Games through Interactioncited by 43
  7. 2019Active Learning for Graph Neural Networks via Node Feature Propagationcited by 37
  8. 2021NL-EDIT: Correcting semantic parse errors through natural language interactioncited by 33
  9. 2020Leap-Of-Thought: Teaching Pre-Trained Models to Systematically Reason Over Implicit Knowledgecited by 24
  10. 2020Active Imitation Learning with Noisy Guidancecited by 15
  11. 2019Interactive Refinement of Cross-Lingual Word Embeddingscited by 13
  12. 2019Improving Natural Language Interaction with Robots Using Advicecited by 9

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 2.

5 papers.

  1. 2022Towards Faithful Model Explanation in NLP: A Surveycited by 129
  2. 2022On Improving Summarization Factual Consistency from Natural Language Feedbackcited by 38
  3. 2023Improving Grounded Language Understanding in a Collaborative Environment by Interacting with Agents Through Help Feedbackcited by 31
  4. 2023Learning to Simulate Natural Language Feedback for Interactive Semantic Parsingcited by 19
  5. 2022Impact of Feedback Type on Explanatory Interactive Learningcited by 7

Similar, not cited

Close in wording, no citation either way

  1. 2025Reasoning-Grounded Natural Language Explanations for Language Models
  2. 2025CLEAR: Contrasting Textual Feedback with Experts and Amateurs for Reasoning
  3. 2025Self-Critique and Refinement for Faithful Natural Language Explanations
  4. 2023Formal Proofs as Structured Explanations: Proposing Several Tasks on Explainable Natural Language Inference
  5. 2023REFINER: Reasoning Feedback on Intermediate Representations
  6. 2022The Unreliability of Explanations in Few-shot Prompting for Textual Reasoning
  7. 2024Conceptual and Unbiased Reasoning in Language Models
  8. 2025Explainability of Large Language Models: Opportunities and Challenges toward Generating Trustworthy Explanations