mlsanityData to 14 Oct 2026

NICE: CVPR 2023 Challenge on Zero-shot Image Captioning

arXiv 2309.01961 ↗First posted 5 Sep 2023cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

In this report, we introduce NICE (New frontiers for zero-shot Image Captioning Evaluation) project and share the results and outcomes of 2023 challenge. This project is designed to challenge the computer vision community to develop robust image captioning models that advance the state-of-the-art both in terms of accuracy and fairness. Through the challenge, the image captioning models were tested using a new evaluation dataset that includes a large variety of visual concepts from many domains. There was no specific training data provided for the challenge, and therefore the challenge entries were required to adapt to new types of image descriptions that had not been seen during training. This report includes information on the newly proposed NICE dataset, evaluation methods, challenge results, and technical details of top-ranking entries. We expect that the outcomes of the challenge will contribute to the improvement of AI models on various vision-language tasks.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 32 of its 43 references that are in the map. First posted on arXiv 5 Sep 2023.

Builds on

32 of 43 references.

References · 32

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  4. 2016Layer Normalizationcited by 5,669
  5. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  6. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  7. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  8. 2017Billion-scale similarity search with GPUscited by 2,497
  9. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,046
  10. 2022Visual Prompt Tuningcited by 1,681
  11. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,619
  12. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,462
20 more
  1. 2021Improving language models by retrieving from trillions of tokenscited by 1,293
  2. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
  3. 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 824
  4. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 745
  5. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 691
  6. 2016Self-critical Sequence Training for Image Captioningcited by 547
  7. 2022Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Taskscited by 511
  8. 2018nocaps: novel object captioning at scalecited by 444
  9. 2023Towards Stable Test-Time Adaptation in Dynamic Wild Worldcited by 415
  10. 2020TextCaps: a Dataset for Image Captioning with Reading Comprehensioncited by 400
  11. 2021Learning How to Ask: Querying LMs with Mixtures of Soft Promptscited by 377
  12. 2023EcoTTA: Memory-Efficient Continual Test-time Adaptation via Self-distilled Regularizationcited by 123
  13. 2022Retrieval Augmented Classification for Long-Tail Visual Recognitioncited by 106
  14. 2014Deep Captioning with Multimodal Recurrent Neural Networks (m-RNN)cited by 94
  15. 2023Symbol tuning improves in-context learning in language modelscited by 75
  16. 2023Learning Customized Visual Models with Retrieval-Augmented Knowledgecited by 58
  17. 2023Re-ViLM: Retrieval-Augmented Visual Language Model for Zero and Few-Shot Image Captioningcited by 41
  18. 2022Noise-aware Learning from Web-crawled Image-Text Data for Image Captioningcited by 22
  19. 2015Exploring Nearest Neighbor Approaches for Image Captioningcited by 19
  20. 2022Large-Scale Bidirectional Training for Zero-Shot Image Captioningcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 3; 2026: 4.

9 papers.

  1. 2025Foundation Models in Autonomous Driving: A Survey on Scenario Generation and Scenario Analysiscited by 38
  2. 2024Preserving Multi-Modal Capabilities of Pre-trained VLMs for Improving Vision-Linguistic Compositionalitycited by 17
  3. 2024DREAM: Improving Video-Text Retrieval Through Relevance-Based Augmentation Using Large Foundation Modelscited by 3
  4. 2026Retrieval-Augmented Generation Based Nurse Observation Extractioncited by 1
  5. 2026Indexing Multimodal Language Models for Large-scale Image Retrievalcited by 1
  6. 2025Group-based Distinctive Image Captioning with Memory Difference Encoding and Attentioncited by 0
  7. 2025Multilingual Training-Free Remote Sensing Image Captioningcited by 0
  8. 2026Caption Generation for Dongba Paintings via Prompt Learning and Semantic Fusioncited by 0
  9. 2026Team MKC at CLPsych 2026: Capturing and Characterizing Mental Health Changes through Social Media Timeline Dynamicscited by 0