mlsanityData to 7 Oct 2026

LLMRefine: Pinpointing and Refining Large Language Models via Fine-Grained Actionable Feedback

arXiv 2311.09336 ↗First posted 15 Nov 2023cs.CLcited by 39

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

Recent large language models (LLM) are leveraging human feedback to improve their generation quality. However, human feedback is costly to obtain, especially during inference. In this work, we propose LLMRefine, an inference time optimization method to refine LLM's output. The core idea is to use a learned fine-grained feedback model to pinpoint defects and guide LLM to refine them iteratively. Using original LLM as a proposal of edits, LLMRefine searches for defect-less text via simulated annealing, trading off the exploration and exploitation. We conduct experiments on three text generation tasks, including machine translation, long-form question answering (QA), and topical summarization. LLMRefine consistently outperforms all baseline approaches, achieving improvements up to 1.7 MetricX points on translation tasks, 8.1 ROUGE-L on ASQA, 2.2 ROUGE-L on topical summarization.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 19 of its 33 references that are in the map. First posted on arXiv 15 Nov 2023.

Builds on

19 of 33 references.

References · 19

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  4. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  5. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  6. 2020COMET: A Neural Framework for MT Evaluationcited by 1,024
  7. 2022Large Language Models Can Self-Improvecited by 656
  8. 2023Fine-Grained Human Feedback Gives Better Rewards for Language Model Trainingcited by 399
  9. 2022Self-critiquing models for assisting human evaluatorscited by 372
  10. 2021Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translationcited by 333
  11. 2023xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detectioncited by 260
  12. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 236
7 more
  1. 2022ASQA: Factoid Questions Meet Long-Form Answerscited by 231
  2. 2022Making Large Language Models Better Reasoners with Step-Aware Verifiercited by 231
  3. 2022CometKiwi: IST-Unbabel 2022 Submission for the Quality Estimation Shared Taskcited by 205
  4. 2021High Quality Rather than High Model Probability: Minimum Bayes Risk Decoding with Neural Metricscited by 104
  5. 2023The Devil is in the Errors: Leveraging Large Language Models for Fine-grained Machine Translation Evaluationcited by 95
  6. 2022Not All Errors are Equal: Learning Text Generation Metrics using Stratified Error Synthesiscited by 18
  7. 2022SESCORE2: Learning Text Generation Evaluation via Synthesizing Realistic Mistakescited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 13; 2025: 21; 2026: 4.

39 papers.

Sort:

By area:

  1. 2025VersaPRM: Multi-Domain Process Reward Model via Synthetic Reasoning Datacited by 39
  2. 2023DSPy Assertions: Computational Constraints for Self-Refining Language Model Pipelinescited by 27
  3. 2024Learning to Refine with Fine-Grained Natural Language Feedbackcited by 27
  4. 2024DelTA: An Online Document-Level Translation Agent Based on Multi-Level Memorycited by 27
  5. 2024Guiding Large Language Models to Post-Edit Machine Translation with Error Annotationscited by 22
  6. 2024MQM-APE: Toward High-Quality Error Annotation Predictors with Automatic Post-Editing in LLM Translation Evaluatorscited by 22
  7. 2025ReviewEval: An Evaluation Framework for AI-Generated Reviewscited by 22
  8. 2024xTower: A Multilingual LLM for Explaining and Correcting Translation Errorscited by 17
  9. 2025Compositional Translation: A Novel LLM-based Approach for Low-resource Machine Translationcited by 13
  10. 2025No Need for Explanations: LLMs can implicitly learn from mistakes in-contextcited by 10
  11. 2025MAMM-Refine: A Recipe for Improving Faithfulness in Generation with Multi-Agent Collaborationcited by 9
  12. 2024Visual Prompting with Iterative Refinement for Design Critique Generationcited by 5
27 more
  1. 2025Two Intermediate Translations Are Better Than One: Fine-tuning LLMs for Document-level Translation Refinementcited by 5
  2. 2024Localizing and Mitigating Errors in Long-form Question Answeringcited by 4
  3. 2024Beyond Human-Only: Evaluating Human-Machine Collaboration for Collecting High-Quality Translation Datacited by 4
  4. 2025Automatic Input Rewriting Improves Translation with Large Language Modelscited by 4
  5. 2025Alleviating Distribution Shift in Synthetic Data for Machine Translation Quality Estimationcited by 4
  6. 2025ReFeed: Multi-dimensional Summarization Refinement with Reflective Reasoning on Feedbackcited by 4
  7. 2025EyeMulator: Improving Code Language Models by Mimicking Human Visual Attentioncited by 4
  8. 2026EvoIdeator: Evolving Scientific Ideas through Checklist-Grounded Reinforcement Learningcited by 4
  9. 2025Decision-Making with Deliberation: Meta-reviewing as a Document-grounded Dialoguecited by 3
  10. 2024Uncovering Factor Level Preferences to Improve Human-Model Alignmentcited by 2
  11. 2024From Jack of All Trades to Master of One: Specializing LLM-based Autoraters to a Test Setcited by 2
  12. 2025SkillVerse : Assessing and Enhancing LLMs with Tree Evaluationcited by 2
  13. 2025Reranking-based Generation for Unbiased Perspective Summarizationcited by 2
  14. 2025Plug-and-Play Dramaturge: A Divide-and-Conquer Approach for Iterative Narrative Script Refinement via Collaborative LLM Agentscited by 2
  15. 2025Adaptive Multi-Agent Response Refinement in Conversational Systemscited by 2
  16. 2026What Does LLM Refinement Actually Improve? A Systematic Study on Document-Level Literary Translationcited by 2
  17. 2024In-context Demonstration Matters: On Prompt Optimization for Pseudo-Supervision Refinementcited by 1
  18. 2024Improving Next Tokens via Second-to-Last Predictions with Generate and Refinecited by 1
  19. 2024Bel Esprit: Multi-Agent Framework for Building AI Model Pipelinescited by 1
  20. 2025Toward Machine Translation Literacy: How Lay Users Perceive and Rely on Imperfect Translationscited by 1
  21. 2025Remedy-R: Generative Reasoning for Machine Translation Evaluation without Error Annotationscited by 1
  22. 2026LongSumEval: Question-Answering Based Evaluation and Feedback-Driven Refinement for Long Document Summarizationcited by 1
  23. 2025Is the Top Still Spinning? Evaluating Subjectivity in Narrative Understandingcited by 0
  24. 2025Interpretable Style Takagi-Sugeno-Kang Fuzzy Clusteringcited by 0
  25. 2025TEN: Table Explicitization, Neurosymbolicallycited by 0
  26. 2025Towards Automated Error Discovery: A Study in Conversational AIcited by 0
  27. 2026OBJECTION! Lawyer Agents Mitigate Guilty Bias in Legal Judgment Predictioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024A Survey on Efficient Inference for Large Language Models
  2. 2025Towards Efficient Multi-LLM Inference: Characterization and Analysis of LLM Routing and Hierarchical Techniques
  3. 2025Bridging the Linguistic Divide: A Survey on Leveraging Large Language Models for Machine Translation
  4. 2023Improving Language Models via Plug-and-Play Retrieval Feedback
  5. 2024CERET: Cost-Effective Extrinsic Refinement for Text Generation
  6. 2025Learning from Self Critique and Refinement for Faithful LLM Summarization
  7. 2024Beyond the Black Box: A Statistical Model for LLM Reasoning and Inference
  8. 2024Learning to Summarize from LLM-generated Feedback