mlsanityData to 14 Oct 2026

VC-Inspector: Advancing Reference-free Evaluation of Video Captions with Factual Analysis

arXiv 2509.16538 ↗First posted 20 Sep 2025cs.CV cs.CLcited by 4

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

We propose VC-Inspector, a lightweight, open-source large multimodal model (LMM) for reference-free evaluation of video captions, with a focus on factual accuracy. Unlike existing metrics that suffer from limited context handling, weak factuality assessment, or reliance on proprietary services, VC-Inspector offers a reproducible and fact-aware alternative that aligns closely with human judgments. To enable robust training and interpretable evaluation, we introduce a systematic framework for generating captions with controllable factual errors, paired with graded quality scores and explanatory annotations. Experiments demonstrate that VC-Inspector achieves state-of-the-art correlation with human judgments, generalizing across diverse domains (e.g., VATEX-Eval, Flickr8K-Expert, and Flickr8K-CF benchmarks) and revealing the potential for caption improvement. Project page is available at https://dipta007.github.io/VC-Inspector

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 28 of its 42 references that are in the map. First posted on arXiv 20 Sep 2025.

Builds on

28 of 42 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025CAPability: A Comprehensive Visual Caption Benchmark for Evaluating Both Correctness and Thoroughness19 Feb
  2. 2024Vision Language Model-based Caption Evaluation Method Leveraging Visual Context Extraction28 Feb

Other references · 26

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  4. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  5. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  6. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
  7. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,046
  8. 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 824
  9. 2017Dense-Captioning Events in Videoscited by 793
  10. 2017Towards Automatic Learning of Procedures from Web Instructional Videoscited by 619
  11. 2020Multi-modal Transformer for Video Retrievalcited by 395
  12. 2021End-to-End Dense Video Captioning with Parallel Decodingcited by 133
14 more
  1. 2024Momentor: Advancing Video Large Language Model with Fine-Grained Temporal Reasoningcited by 122
  2. 2017FOIL it! Find One mismatch between Image and Language captioncited by 95
  3. 2023Positive-Augmented Contrastive Learning for Image and Video Captioning Evaluationcited by 78
  4. 2019TIGEr: Text-to-Image Grounding for Image Caption Evaluationcited by 58
  5. 2018Learning to Evaluate Image Captioningcited by 53
  6. 2023FACTUAL: A Benchmark for Faithful and Consistent Textual Scene Graph Parsingcited by 50
  7. 2024Polos: Multimodal Metric Learning from Human Feedback for Image Captioningcited by 46
  8. 2021UMIC: An Unreferenced Metric for Image Captioning via Contrastive Learningcited by 37
  9. 2021EMScore: Evaluating Video Captioning via Coarse-Grained and Fine-Grained Embedding Matchingcited by 36
  10. 2024G-VEval: A Versatile Metric for Evaluating Image and Video Captions Using GPT-4ocited by 23
  11. 2019VIFIDEL: Evaluating the Visual Fidelity of Image Descriptionscited by 21
  12. 2023Models See Hallucinations: Evaluating the Factuality in Video Captioningcited by 19
  13. 2024Positive-Augmented Contrastive Learning for Vision-and-Language Evaluation and Trainingcited by 16
  14. 2024HICEScore: A Hierarchical Metric for Image Captioning Evaluationcited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Many Dialects, Many Languages, One Cultural Lens: Evaluating Multilingual VLMs for Bengali Culture Understanding Across Historically Linked Languages and Regional Dialectscited by 3
  2. 2026OracleZoom: On-Policy Self-Distillation Inspired Reference-Constrained Recursive Image Super Resolutioncited by 1
  3. 2026BEFORE THE FLIP: Measuring Hidden Score Shifts In Quantized Vision Language Models Before The Answer Changes for Visual Question Answeringcited by 1
  4. 2026New Evidence, Same Choice: Testing Physical Experiment Selection in Vision Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Unblocking Fine-Grained Evaluation of Detailed Captions: An Explaining AutoRater and Critic-and-Revise Pipeline
  2. 2025VCapsBench: A Large-scale Fine-grained Benchmark for Video Caption Quality Evaluation
  3. 2025OVFact: Measuring and Improving Open-Vocabulary Factuality for Long Caption Models
  4. 2025IF-VidCap: Can Video Caption Models Follow Instructions?
  5. 2024Benchmarking and Improving Detail Image Caption
  6. 2025CapArena: Benchmarking and Analyzing Detailed Image Captioning in the LLM Era
  7. 2026DualFact+: A Multimodal Fact Verification Framework for Procedural Video Understanding
  8. 2024FLEUR: An Explainable Reference-Free Evaluation Metric for Image Captioning Using a Large Multimodal Model