mlsanityData to 7 Oct 2026

IC3: Image Captioning by Committee Consensus

arXiv 2302.01328 ↗First posted 2 Feb 2023cs.CV cs.AI cs.CLcited by 20

Not on the map. Why?

Abstract (arXiv)

If you ask a human to describe an image, they might do so in a thousand different ways. Traditionally, image captioning models are trained to generate a single "best" (most like a reference) image caption. Unfortunately, doing so encourages captions that are "informationally impoverished," and focus on only a subset of the possible details, while ignoring other potentially useful information in the scene. In this work, we introduce a simple, yet novel, method: "Image Captioning by Committee Consensus" (IC3), designed to generate a single caption that captures high-level details from several annotator viewpoints. Humans rate captions produced by IC3 at least as helpful as baseline SOTA models more than two thirds of the time, and IC3 can improve the performance of SOTA automated recall systems by up to 84%, outperforming single human-generated reference captions, and indicating significant improvements over SOTA approaches for visual description. Code is available at https://davidmchan.github.io/caption-by-committee/

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Builds on 57 of its 99 references that are in the map. First posted on arXiv 2 Feb 2023.

Builds on

57 of 99 references.

References · 57

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  6. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  7. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  8. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  9. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  10. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  11. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  12. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
38 more
  1. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  2. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,265
  3. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  4. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 762
  5. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 742
  6. 2021GPT Understands, Toocited by 715
  7. 2021ClipCap: CLIP Prefix for Image Captioningcited by 557
  8. 2022Socratic Models: Composing Zero-Shot Multimodal Reasoning with Languagecited by 513
  9. 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 484
  10. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 466
  11. 2022News Summarization and Evaluation in the Era of GPT-3cited by 324
  12. 2016Deep Reinforcement Learning for Dialogue Generationcited by 286
  13. 2017Style Transfer from Non-Parallel Text by Cross-Alignmentcited by 246
  14. 2015DenseCap: Fully Convolutional Localization Networks for Dense Captioningcited by 176
  15. 2016A Hierarchical Approach for Generating Descriptive Image Paragraphscited by 104
  16. 2022PromptCap: Prompt-Guided Task-Aware Image Captioningcited by 104
  17. 2017Towards a Visual Privacy Advisor: Understanding and Predicting Privacy Risks in Imagescited by 101
  18. 2022Plug-and-Play VQA: Zero-shot VQA by Conjoining Large Pretrained Models with Zero Trainingcited by 88
  19. 2021Medically Aware GPT-3 as a Data Generator for Medical Dialogue Summarizationcited by 76
  20. 2017Towards Diverse and Natural Image Descriptions via a Conditional GANcited by 59
  21. 2017Text Summarization Techniques: A Brief Surveycited by 59
  22. 2018Fast, Diverse and Accurate Image Captioning Guided By Part-of-Speechcited by 55
  23. 2016Semantic Compositional Networks for Visual Captioningcited by 50
  24. 2019Describing like humans: on diversity in image captioningcited by 46
  25. 2021Injecting Semantic Concepts into End-to-End Image Captioningcited by 41
  26. 2016Dense Captioning with Joint Inference and Visual Contextcited by 40
  27. 2021Transparent Human Evaluation for Image Captioningcited by 40
  28. 2019Dense Relational Captioning: Triple-Stream Networks for Relationship-Based Captioningcited by 35
  29. 2017Diverse and Accurate Image Description Using a Variational Auto-Encoder with an Additive Gaussian Encoding Spacecited by 33
  30. 2019Context and Attribute Grounded Dense Captioningcited by 31
  31. 2019Context-Aware Visual Policy Network for Fine-Grained Image Captioningcited by 29
  32. 2020Curious Case of Language Generation Evaluation Metrics: A Cautionary Talecited by 28
  33. 2017Recurrent Topic-Transition GAN for Visual Paragraph Generationcited by 25
  34. 2020Automatic Text Summarization of COVID-19 Medical Research Articles using BERT and GPT-2cited by 23
  35. 2020Diverse Image Captioning with Context-Object Split Latent Spacescited by 22
  36. 2019Sequential Latent Spaces for Modeling the Intention During Diverse Image Captioningcited by 21
  37. 2022Visual Clues: Bridging Vision and Language Foundations for Image Paragraph Captioningcited by 21
  38. 2019Quality Estimation for Image Captions Based on Large-scale Human Evaluationscited by 17

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 10; 2025: 5; 2026: 2.

20 papers.

Sort:

By area:

  1. 2024ALOHa: A New Measure for Hallucination in Captioning Modelscited by 44
  2. 2023Segment and Caption Anythingcited by 30
  3. 2024Visual Fact Checker: Enabling High-Fidelity Detailed Caption Generationcited by 28
  4. 2024VLRM: Vision-Language Models act as Reward Models for Image Captioningcited by 18
  5. 2023Guiding Image Captioning Models Toward More Specific Captionscited by 17
  6. 2024Large Language Model Based Generative Error Correction: A Challenge and Baselines for Speech Recognition, Speaker Tagging, and Emotion Recognitioncited by 10
  7. 2024From Descriptive Richness to Bias: Unveiling the Dark Side of Generative Image Caption Enrichmentcited by 8
  8. 2025Enhancing Remote Sensing Vision-Language Models Through MLLM and LLM-Based High-Quality Image-Text Dataset Generationcited by 8
  9. 2024Self-Taught Recognizer: Toward Unsupervised Adaptation for Speech Foundation Modelscited by 7
  10. 2024Toward Automatic Relevance Judgment using Vision--Language Models for Image--Text Retrieval Evaluationcited by 5
  11. 2024Revisiting Image Captioning Training Paradigm via Direct CLIP-based Optimizationcited by 5
  12. 2025"It's trained by non-disabled people": Evaluating How Image Quality Affects Product Captioning with Vision-Language Modelscited by 3
8 more
  1. 2025SpeechIQ: Speech-Agentic Intelligence Quotient Across Cognitive Levels in Voice Understanding by Large Language Modelscited by 2
  2. 2025LOTUS: A Leaderboard for Detailed Image Captioning from Quality to Societal Bias and User Preferencescited by 2
  3. 2023A Comprehensive Analysis of Real-World Image Captioning and Scene Identificationcited by 1
  4. 2025Embodied Image Captioning: Self-supervised Learning Agents for Spatially Coherent Image Descriptionscited by 1
  5. 2024Surveying the Landscape of Image Captioning Evaluation: A Comprehensive Taxonomy, Trends and Metrics Analysiscited by 0
  6. 2024Fluent and Accurate Image Captioning with a Self-Trained Reward Modelcited by 0
  7. 2026Towards Clinically Faithful Medical Image Captioning via Enhanced Vision-Language Alignmentcited by 0
  8. 2026When Words Speak Louder than Images: Towards Understanding Language Bias in Vision-Language Modelscited by 0