mlsanityData to 14 Oct 2026

Retrieval Enhanced Feedback via In-context Neural Error-book

arXiv 2508.16313 ↗First posted 22 Aug 2025cs.LG cs.AI cs.CLcited by 0

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Recent advancements in Large Language Models (LLMs) have significantly improved reasoning capabilities, with in-context learning (ICL) emerging as a key technique for adaptation without retraining. While previous works have focused on leveraging correct examples, recent research highlights the importance of learning from errors to enhance performance. However, existing methods lack a structured framework for analyzing and mitigating errors, particularly in Multimodal Large Language Models (MLLMs), where integrating visual and textual inputs adds complexity. To address this issue, we propose REFINE: Retrieval-Enhanced Feedback via In-context Neural Error-book, a teacher-student framework that systematically structures errors and provides targeted feedback. REFINE introduces three systematic queries to construct structured feedback -- Feed-Target, Feed-Check, and Feed-Path -- to enhance multimodal reasoning by prioritizing relevant visual information, diagnosing critical failure points, and formulating corrective actions. Unlike prior approaches that rely on redundant retrievals, REFINE optimizes structured feedback retrieval, improving inference efficiency, token usage, and scalability. Our results demonstrate substantial speedup, reduced computational costs, and successful generalization, highlighting REFINE's potential for enhancing multimodal reasoning.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 20 of its 37 references that are in the map. First posted on arXiv 22 Aug 2025.

Builds on

20 of 37 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Can LLMs Learn from Previous Mistakes? Investigating LLMs' Errors to Boost for Reasoning29 Mar
  2. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?25 Feb

Other references · 18

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2024VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Modelscited by 455
  4. 2023Larger language models do in-context learning differentlycited by 394
  5. 2024MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?cited by 292
  6. 2024VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documentscited by 245
  7. 2025MLLMs Know Where to Look: Training-free Perception of Small Visual Details with Multimodal LLMscited by 176
  8. 2023MMICL: Empowering Vision-language Model with Multi-Modal In-Context Learningcited by 172
  9. 2024SEED-Bench-2-Plus: Benchmarking Multimodal Large Language Models with Text-Rich Visual Comprehensioncited by 146
  10. 2024The Revolution of Multimodal Large Language Models: A Surveycited by 137
  11. 2023Waffling around for Performance: Visual Classification with Random Words and Broad Conceptscited by 101
  12. 2024SPIQA: A Dataset for Multimodal Question Answering on Scientific Paperscited by 75
6 more
  1. 2024In-Context Principle Learning from Mistakescited by 44
  2. 2024Stepwise Verification and Remediation of Student Reasoning Errors with Large Language Model Tutorscited by 41
  3. 2024MLLM-CompBench: A Comparative Reasoning Benchmark for Multimodal LLMscited by 32
  4. 2023Probabilistic Tree-of-thought Reasoning for Answering Knowledge-intensive Complex Questionscited by 19
  5. 2023Follow-Up Differential Descriptions: Language Models Resolve Ambiguities for Image Classificationcited by 13
  6. 2024RAVEN: Multitask Retrieval Augmented Vision-Language Learningcited by 10

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025MMRefine: Unveiling the Obstacles to Robust Refinement in Multimodal Large Language Models
  2. 2023MAF: Multi-Aspect Feedback for Improving Reasoning in Large Language Models
  3. 2024Does In-Context Learning Really Learn? Rethinking How Large Language Models Respond and Solve Tasks via In-Context Learning
  4. 2023Improving Language Models via Plug-and-Play Retrieval Feedback
  5. 2023Self-Refine: Iterative Refinement with Self-Feedback
  6. 2026MMPCBench: Benchmarking Multimodal Large Language Models on Proactive Critique of Flawed Inputs
  7. 2023Beyond Task Performance: Evaluating and Reducing the Flaws of Large Multimodal Models with In-Context Learning
  8. 2025Can Large Multimodal Models Actively Recognize Faulty Inputs? A Systematic Evaluation Framework of Their Input Scrutiny Ability