# G-VEval: A Versatile Metric for Evaluating Image and Video Captions Using GPT-4o

arXiv: https://arxiv.org/abs/2412.13647 · HTML: https://mlsanity.com/paper/2412.13647/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 23 references that are in the map.
- First posted on arXiv 18 Dec 2024.

## Builds on (most cited first, 10 of 10)

- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding](https://mlsanity.com/paper/2306.02858/) (2023)
- [Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks](https://mlsanity.com/paper/2004.06165/) (2020)
- [Positive-Augmented Contrastive Learning for Image and Video Captioning Evaluation](https://mlsanity.com/paper/2303.12112/) (2023)
- [Learning to Evaluate Image Captioning](https://mlsanity.com/paper/1806.06422/) (2018)
- [Polos: Multimodal Metric Learning from Human Feedback for Image Captioning](https://mlsanity.com/paper/2402.18091/) (2024)
- [EMScore: Evaluating Video Captioning via Coarse-Grained and Fine-Grained Embedding Matching](https://mlsanity.com/paper/2111.08919/) (2021)

## Built on by (most cited first, 23 of 23)

- [Are Video Models Ready as Zero-Shot Reasoners? An Empirical Study with the MME-CoF Benchmark](https://mlsanity.com/paper/2510.26802/) (2025)
- [Vamba: Understanding Hour-Long Videos with Hybrid Mamba-Transformers](https://mlsanity.com/paper/2503.11579/) (2025)
- [RSCC: A Large-Scale Remote Sensing Change Caption Dataset for Disaster Events](https://mlsanity.com/paper/2509.01907/) (2025)
- [JEEM: Vision-Language Understanding in Four Arabic Dialects](https://mlsanity.com/paper/2503.21910/) (2025)
- [Evaluation of Multilingual Image Captioning: How far can we get with CLIP models?](https://mlsanity.com/paper/2502.06600/) (2025)
- [VELA: An LLM-Hybrid-as-a-Judge Approach for Evaluating Long Image Captions](https://mlsanity.com/paper/2509.25818/) (2025)
- [DynamicVerse: A Physically-Aware Multimodal Framework for 4D World Modeling](https://mlsanity.com/paper/2512.03000/) (2025)
- [VC-Inspector: Advancing Reference-free Evaluation of Video Captions with Factual Analysis](https://mlsanity.com/paper/2509.16538/) (2025)
- [Multi-modal, Multi-task, Multi-criteria Automatic Evaluation with Vision Language Models](https://mlsanity.com/paper/2412.14613/) (2024)
- [FinCap: Topic-Aligned Captions for Short-Form Financial YouTube Videos](https://mlsanity.com/paper/2509.25745/) (2025)
- [DISCODE: Distribution-Aware Score Decoder for Robust Automatic Evaluation of Image Captioning](https://mlsanity.com/paper/2512.14420/) (2025)
- [LLM-Free Image Captioning Evaluation in Reference-Flexible Settings](https://mlsanity.com/paper/2512.21582/) (2025)
- [ColorFLUX: A Structure-Color Decoupling Framework for Old Photo Colorization](https://mlsanity.com/paper/2603.28162/) (2026)
- [Are Large Vision-Language Models Ready to Guide Blind and Low-Vision Individuals?](https://mlsanity.com/paper/2510.00766/) (2025)
- [MLLM-as-a-Judge Exhibits Model Preference Bias](https://mlsanity.com/paper/2604.11589/) (2026)
- [Rigel: Self-Distilled Score Adaptation for Image and Video Captioning Evaluation](https://mlsanity.com/paper/2606.29997/) (2026)
- [Redemption Score: A Multi-Modal Evaluation Framework for Image Captioning via Distributional, Perceptual, and Linguistic Signal Triangulation](https://mlsanity.com/paper/2505.16180/) (2025)
- [A Critical Study of Automatic Evaluation in Sign Language Translation](https://mlsanity.com/paper/2510.25434/) (2025)
- [ITIScore: An Image-to-Text-to-Image Rating Framework for the Image Captioning Ability of MLLMs](https://mlsanity.com/paper/2604.03765/) (2026)
- [Vision-language Models for Driver Monitoring Systems: A Driver Activity Description Dataset](https://mlsanity.com/paper/2606.02273/) (2026)
- [DiCoBench: Benchmarking Multi-Image Fine-Grained Perception via Differential and Commonality Visual Cues](https://mlsanity.com/paper/2606.26602/) (2026)
- [CLIP-CC-Bench: Evaluating Paragraph-Level Video Descriptions in Video-Language Models](https://mlsanity.com/paper/2608.04302/) (2026)
- [Spot-the-shift: Evaluating Grounded Image Difference Captioning of Long-term Changes](https://mlsanity.com/paper/2609.10356/) (2026)

## Abstract (arXiv)

Evaluation metric of visual captioning is important yet not thoroughly explored. Traditional metrics like BLEU, METEOR, CIDEr, and ROUGE often miss semantic depth, while trained metrics such as CLIP-Score, PAC-S, and Polos are limited in zero-shot scenarios. Advanced Language Model-based metrics also struggle with aligning to nuanced human preferences. To address these issues, we introduce G-VEval, a novel metric inspired by G-Eval and powered by the new GPT-4o. G-VEval uses chain-of-thought reasoning in large multimodal models and supports three modes: reference-free, reference-only, and combined, accommodating both video and image inputs. We also propose MSVD-Eval, a new dataset for video captioning evaluation, to establish a more transparent and consistent framework for both human experts and evaluation metrics. It is designed to address the lack of clear criteria in existing datasets by introducing distinct dimensions of Accuracy, Completeness, Conciseness, and Relevance (ACCR). Extensive results show that G-VEval outperforms existing methods in correlation with human annotations, as measured by Kendall tau-b and Kendall tau-c. This provides a flexible solution for diverse captioning tasks and suggests a straightforward yet effective approach for large language models to understand video content, paving the way for advancements in automated captioning. Codes are available at https://github.com/ztangaj/gveval