# Learning to Correction: Explainable Feedback Generation for Visual Commonsense Reasoning Distractor

arXiv: https://arxiv.org/abs/2412.07801 · HTML: https://mlsanity.com/paper/2412.07801/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 0 of its 54 references that are in the map.
- First posted on arXiv 8 Dec 2024.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)
- Multimodal LLMs / vision-language models / [VLM benchmarks and visual reasoning](https://mlsanity.com/mllm/vlm-benchmarks-and-visual-reasoning/)

## Built on by (most cited first, 7 of 7)

- [UORA: Uniform Orthogonal Reinitialization Adaptation in Parameter-Efficient Fine-Tuning of Large Models](https://mlsanity.com/paper/2505.20154/) (2025)
- [Collaborative Multi-LoRA Experts with Achievement-based Multi-Tasks Loss for Unified Multimodal Information Extraction](https://mlsanity.com/paper/2505.06303/) (2025)
- [RPO: Fine-Tuning Visual Generative Models via Rich Vision-Language Preferences](https://mlsanity.com/paper/2503.11720/) (2025)
- [CADReview: Automatically Reviewing CAD Programs with Error Detection and Correction](https://mlsanity.com/paper/2505.22304/) (2025)
- [ExpStar: Towards Automatic Commentary Generation for Multi-discipline Scientific Experiments](https://mlsanity.com/paper/2507.09693/) (2025)
- [Classic4Children: Adapting Chinese Literary Classics for Children with Large Language Model](https://mlsanity.com/paper/2502.01090/) (2025)
- [SciGen-Verifier: A Multimodal Reasoner for Explainable Verification in Scientific Image Generation](https://mlsanity.com/paper/2609.33399/) (2026)

## Abstract (arXiv)

Large multimodal models (LMMs) have shown remarkable performance in the visual commonsense reasoning (VCR) task, which aims to answer a multiple-choice question based on visual commonsense within an image. However, the ability of LMMs to correct potential visual commonsense errors in the distractor upon their occurrence is yet under-explored. Drawing inspiration from how a human teacher crafts challenging distractors to test students' comprehension of the concepts or skills and assists them in identifying and correcting errors toward the answer, we are the pioneering research for LMMs to simulate this error correction process. To this end, we employ GPT-4 as a ``teacher'' to collect the explainable feedback dataset VCR-DF for error correction, which serves as a benchmark to evaluate the ability of LMMs to identify misconceptions and clarify reasons behind the error in VCR distractors toward final answers. In addition, we propose an LMM-based Pedagogical Expert Instructed Feedback Generation (PEIFG) model to incorporate the learnable expert prompts and multimodal instruction as guidance for feedback generation. Experimental results show that our PEIFG significantly outperforms existing LMMs. We believe that our benchmark provides a new direction for evaluating the capabilities of LMMs.