mlsanityData to 14 Oct 2026

CIDEr: Consensus-based Image Description Evaluation

arXiv 1411.5726 ↗First posted 20 Nov 2014cs.CV cs.CL cs.IRcited by 2,046

Not on the map. Why?

Cited by 413 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Automatically describing an image with a sentence is a long-standing challenge in computer vision and natural language processing. Due to recent progress in object detection, attribute classification, action recognition, etc., there is renewed interest in this area. However, evaluating the quality of descriptions has proven to be challenging. We propose a novel paradigm for evaluating image descriptions that uses human consensus. This paradigm consists of three main parts: a new triplet-based method of collecting human annotations to measure consensus, a new automated metric (CIDEr) that captures consensus, and two new datasets: PASCAL-50S and ABSTRACT-50S that contain 50 sentences describing each image. Our simple metric captures human judgment of consensus better than existing metrics across sentences generated by various sources. We also evaluate five state-of-the-art image description approaches using this new protocol and provide a benchmark for future comparisons. A version of CIDEr named CIDEr-D is available as a part of MS COCO evaluation server to enable systematic evaluation and benchmarking.

Key facts. Cited by 2,046 papers in the map (data through 14 Oct 2026). First posted on arXiv 20 Nov 2014.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2017: 1; 2018: 0; 2019: 1; 2020: 1; 2021: 249; 2022: 296; 2023: 412; 2024: 444; 2025: 387; 2026: 255.

2,046 papers.

Sort:

By area:

  1. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  2. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  3. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  4. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  5. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
  6. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
  7. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
  8. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
  9. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
  10. 2023A Survey on Multimodal Large Language Modelscited by 909
  11. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 766
  12. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 745
38 more
  1. 2024An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Modelscited by 642
  2. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 608
  3. 2023OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Modelscited by 565
  4. 2021ClipCap: CLIP Prefix for Image Captioningcited by 559
  5. 2023Language Is Not All You Need: Aligning Perception with Language Modelscited by 543
  6. 2022GIT: A Generative Image-to-text Transformer for Vision and Languagecited by 537
  7. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 536
  8. 2023MotionGPT: Human Motion as a Foreign Languagecited by 527
  9. 2022Socratic Models: Composing Zero-Shot Multimodal Reasoning with Languagecited by 513
  10. 2023VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Taskscited by 501
  11. 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 483
  12. 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Modelcited by 461
  13. 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuningcited by 445
  14. 2023TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answeringcited by 428
  15. 2021Unifying Vision-and-Language Tasks via Text Generationcited by 412
  16. 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understandingcited by 406
  17. 2024Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teacherscited by 393
  18. 2022TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Textscited by 359
  19. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 358
  20. 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 346
  21. 2021How Much Can CLIP Benefit Vision-and-Language Tasks?cited by 321
  22. 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Modelscited by 316
  23. 2023Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Actioncited by 302
  24. 2023DreamLLM: Synergistic Multimodal Comprehension and Creationcited by 297
  25. 2023Towards Generalist Biomedical AIcited by 289
  26. 2023LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understandingcited by 274
  27. 2023WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Researchcited by 267
  28. 2023Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioningcited by 263
  29. 2022Diffsound: Discrete Diffusion Model for Text-to-sound Generationcited by 257
  30. 2023VTimeLLM: Empower LLM to Grasp Video Momentscited by 255
  31. 2022Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimizationcited by 246
  32. 2022Transformers in Medical Imaging: A Surveycited by 245
  33. 2021QuestEval: Summarization Asks for Fact-based Evaluationcited by 244
  34. 2023DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Drivingcited by 233
  35. 2024MM1: Methods, Analysis & Insights from Multimodal LLM Pre-trainingcited by 232
  36. 2022DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Modelscited by 230
  37. 2025Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoningcited by 230
  38. 2024MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understandingcited by 229

50 most cited of 2,046 All 2,046 (JSON)