# Evaluating Image Caption via Cycle-consistent Text-to-Image Generation

arXiv: https://arxiv.org/abs/2501.03567 · HTML: https://mlsanity.com/paper/2501.03567/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jan 2025.

## Built on by (most cited first, 4 of 4)

- [Unifying Vision-Language Latents for Zero-label Image Caption Enhancement](https://mlsanity.com/paper/2510.12931/) (2025)
- [Redemption Score: A Multi-Modal Evaluation Framework for Image Captioning via Distributional, Perceptual, and Linguistic Signal Triangulation](https://mlsanity.com/paper/2505.16180/) (2025)
- [CycleCap: Improving VLMs Captioning Performance via Self-Supervised Cycle Consistency Fine-Tuning](https://mlsanity.com/paper/2603.18282/) (2026)
- [A Reconstruction-Based Framework for Caption Evaluation Beyond Reference Captions](https://mlsanity.com/paper/2607.23235/) (2026)

## Abstract (arXiv)

Evaluating image captions typically relies on reference captions, which are costly to obtain and exhibit significant diversity and subjectivity. While reference-free evaluation metrics have been proposed, most focus on cross-modal evaluation between captions and images. Recent research has revealed that the modality gap generally exists in the representation of contrastive learning-based multi-modal systems, undermining the reliability of cross-modality metrics like CLIPScore. In this paper, we propose CAMScore, a cyclic reference-free automatic evaluation metric for image captioning models. To circumvent the aforementioned modality gap, CAMScore utilizes a text-to-image model to generate images from captions and subsequently evaluates these generated images against the original images. Furthermore, to provide fine-grained information for a more comprehensive evaluation, we design a three-level evaluation framework for CAMScore that encompasses pixel-level, semantic-level, and objective-level perspectives. Extensive experiment results across multiple benchmark datasets show that CAMScore achieves a superior correlation with human judgments compared to existing reference-based and reference-free metrics, demonstrating the effectiveness of the framework.