mlsanityData to 7 Oct 2026

Microsoft COCO Captions: Data Collection and Evaluation Server

arXiv 1504.00325 ↗First posted 1 Apr 2015cs.CV cs.CLcited by 1,447

Not on the map. Why?

Cited by 448 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In this paper we describe the Microsoft COCO Caption dataset and evaluation server. When completed, the dataset will contain over one and a half million captions describing over 330,000 images. For the training and validation images, five independent human generated captions will be provided. To ensure consistency in evaluation of automatic caption generation algorithms, an evaluation server is used. The evaluation server receives candidate captions and scores them using several popular metrics, including BLEU, METEOR, ROUGE and CIDEr. Instructions for using the evaluation server are provided.

Key facts. Cited by 1,447 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Apr 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2017: 1; 2018: 0; 2019: 0; 2020: 0; 2021: 167; 2022: 214; 2023: 314; 2024: 331; 2025: 273; 2026: 147.

1,447 papers.

Sort:

By area:

  1. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  2. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  3. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  4. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  5. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  6. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  7. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  8. 2025Gemma 3 Technical Reportcited by 1,686
  9. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  10. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
  11. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  12. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,265
38 more
  1. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  2. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
  3. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,209
  4. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  5. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,168
  6. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,038
  7. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
  8. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,000
  9. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 950
  10. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 891
  11. 2023VideoChat: Chat-Centric Video Understandingcited by 851
  12. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 847
  13. 2023Vision-Language Models for Vision Tasks: A Surveycited by 834
  14. 2022A Generalist Agentcited by 802
  15. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 776
  16. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 762
  17. 2024Emu3: Next-Token Prediction is All You Needcited by 761
  18. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 742
  19. 2024DoRA: Weight-Decomposed Low-Rank Adaptationcited by 740
  20. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 702
  21. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 696
  22. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 679
  23. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 677
  24. 2021FLAVA: A Foundational Language And Vision Alignment Modelcited by 646
  25. 2021CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrievalcited by 645
  26. 2021SimVLM: Simple Visual Language Model Pretraining with Weak Supervisioncited by 645
  27. 2023DataComp: In search of the next generation of multimodal datasetscited by 635
  28. 2021RegionCLIP: Region-based Language-Image Pretrainingcited by 612
  29. 2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Modelcited by 602
  30. 2021Extract Free Dense Labels from CLIPcited by 580
  31. 2021LiT: Zero-Shot Transfer with Locked-image text Tuningcited by 572
  32. 2022Open-Vocabulary Semantic Segmentation with Mask-adapted CLIPcited by 560
  33. 2023OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Modelscited by 560
  34. 2021ClipCap: CLIP Prefix for Image Captioningcited by 557
  35. 2022Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learningcited by 553
  36. 2023Otter: A Multi-Modal Model with In-Context Instruction Tuningcited by 548
  37. 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Modelscited by 539
  38. 2022Socratic Models: Composing Zero-Shot Multimodal Reasoning with Languagecited by 513

50 most cited of 1,447 All 1,447 (JSON)