# Technical Report of NICE Challenge at CVPR 2024: Caption Re-ranking Evaluation Using Ensembled CLIP and Consensus Scores

arXiv: https://arxiv.org/abs/2405.01028 · HTML: https://mlsanity.com/paper/2405.01028/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 15 references that are in the map.
- First posted on arXiv 2 May 2024.

## Builds on (most cited first, 9 of 9)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [Reproducible scaling laws for contrastive language-image learning](https://mlsanity.com/paper/2212.07143/) (2022)
- [SPICE: Semantic Propositional Image Caption Evaluation](https://mlsanity.com/paper/1607.08822/) (2016)
- [EVA-CLIP: Improved Training Techniques for CLIP at Scale](https://mlsanity.com/paper/2303.15389/) (2023)
- [MobileCLIP: Fast Image-Text Models through Multi-Modal Reinforced Training](https://mlsanity.com/paper/2311.17049/) (2023)

## Built on by (most cited first, 5 of 5)

- [ORIC: Benchmarking Object Recognition under Contextual Incongruity in Large Vision-Language Models](https://mlsanity.com/paper/2509.15695/) (2025)
- [Do Image-Text Metrics Respect Semantic Invariances?](https://mlsanity.com/paper/2605.24702/) (2026)
- [Embodied Image Captioning: Self-supervised Learning Agents for Spatially Coherent Image Descriptions](https://mlsanity.com/paper/2504.08531/) (2025)
- [Memory-Augmented Vision-Language Agents for Persistent and Semantically Consistent Object Captioning](https://mlsanity.com/paper/2603.24257/) (2026)
- [Zero-Shot Captioning for Cultural Heritage: Automated Image Analysis of Traditional Indonesian Clothing](https://mlsanity.com/paper/2606.13275/) (2026)

## Abstract (arXiv)

This report presents the ECO (Ensembled Clip score and cOnsensus score) pipeline from team DSBA LAB, which is a new framework used to evaluate and rank captions for a given image. ECO selects the most accurate caption describing image. It is made possible by combining an Ensembled CLIP score, which considers the semantic alignment between the image and captions, with a Consensus score that accounts for the essentialness of the captions. Using this framework, we achieved notable success in the CVPR 2024 Workshop Challenge on Caption Re-ranking Evaluation at the New Frontiers for Zero-Shot Image Captioning Evaluation (NICE). Specifically, we secured third place based on the CIDEr metric, second in both the SPICE and METEOR metrics, and first in the ROUGE-L and all BLEU Score metrics. The code and configuration for the ECO framework are available at https://github.com/DSBA-Lab/ECO .