mlsanityData to 7 Oct 2026

Contrastive Learning for Image Captioning

arXiv 1710.02534 ↗First posted 6 Oct 2017cs.CVcited by 54

Not on the map. Why?

Abstract (arXiv)

Image captioning, a popular topic in computer vision, has achieved substantial progress in recent years. However, the distinctiveness of natural descriptions is often overlooked in previous work. It is closely related to the quality of captions, as distinctive captions are more likely to describe images with their unique aspects. In this work, we propose a new learning method, Contrastive Learning (CL), for image captioning. Specifically, via two constraints formulated on top of a reference model, the proposed method can encourage distinctiveness, while maintaining the overall quality of the generated captions. We tested our method on two challenging datasets, where it improves the baseline model by significant margins. We also showed in our studies that the proposed method is generic and can be used for models with various structures.

Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Oct 2017.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 12; 2022: 15; 2023: 14; 2024: 10; 2025: 3.

54 papers.

Sort:

By area:

  1. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  2. 2021From Show to Tell: A Survey on Deep Learning-based Image Captioningcited by 137
  3. 2022Does CLIP Bind Concepts? Probing Compositionality in Large Image Modelscited by 101
  4. 2021Contrastive Attention for Automatic Chest X-ray Report Generationcited by 71
  5. 2022Fine-grained Image Captioning with CLIP Rewardcited by 69
  6. 2021Human-like Controllable Image Captioning with Verb-specific Semantic Rolescited by 41
  7. 2021Weakly Supervised Contrastive Learning for Chest X-Ray Report Generationcited by 39
  8. 2021Deep Matching Prior: Test-Time Optimization for Dense Correspondencecited by 30
  9. 2022Modality-Aware Contrastive Instance Learning with Self-Distillation for Weakly-Supervised Audio-Visual Violence Detectioncited by 26
  10. 2022Prompt-based Learning for Unpaired Image Captioningcited by 23
  11. 2022Affection: Learning Affective Explanations for Real-World Visual Datacited by 19
  12. 2024Emergent Visual-Semantic Hierarchies in Image-Text Representationscited by 19
38 more
  1. 2022FCL-GAN: A Lightweight and Real-Time Baseline for Unsupervised Blind Image Deblurringcited by 18
  2. 2023CamDiff: Camouflage Image Augmentation via Diffusion Modelcited by 16
  3. 2022DORA: Exploring Outlier Representations in Deep Neural Networkscited by 14
  4. 2023Cross-Domain Image Captioning with Discriminative Finetuningcited by 14
  5. 2021CIL: Contrastive Instance Learning Framework for Distantly Supervised Relation Extractioncited by 13
  6. 2021Group-based Distinctive Image Captioning with Memory Attentioncited by 13
  7. 2024DistinctAD: Distinctive Audio Description Generation in Contextscited by 11
  8. 2021Learning with Instance Bundles for Reading Comprehensioncited by 8
  9. 2021Bilateral Personalized Dialogue Generation with Contrastive Learningcited by 8
  10. 2022Rethinking the Reference-based Distinctive Image Captioningcited by 8
  11. 2022On Distinctive Image Captioning via Comparing and Reweightingcited by 7
  12. 2023Image Captioning with Multi-Context Synthetic Datacited by 7
  13. 2024Efficient and Versatile Robust Fine-Tuning of Zero-shot Modelscited by 7
  14. 2021FICGAN: Facial Identity Controllable GAN for De-identificationcited by 6
  15. 2022Distinctive Image Captioning via CLIP Guided Group Optimizationcited by 6
  16. 2022CapEnrich: Enriching Caption Semantics for Web Images via Cross-modal Pre-trained Knowledgecited by 5
  17. 2023COMICS: End-to-end Bi-grained Contrastive Learning for Multi-face Forgery Detectioncited by 5
  18. 2023Towards More Faithful Natural Language Explanation Using Multi-Level Contrastive Learning in VQAcited by 5
  19. 2022Unlabeled Data Help: Minimax Analysis and Adversarial Robustnesscited by 4
  20. 2022Switching to Discriminative Image Captioning by Relieving a Bottleneck of Reinforcement Learningcited by 4
  21. 2023Topic-Guided Sampling For Data-Efficient Multi-Domain Stance Detectioncited by 4
  22. 2023Learning Descriptive Image Captioning via Semipermeable Maximum Likelihood Estimationcited by 4
  23. 2023Improving Reference-based Distinctive Image Captioning with Contrastive Rewardscited by 4
  24. 2023TATA: Stance Detection via Topic-Agnostic and Topic-Aware Embeddingscited by 4
  25. 2025ZeroVO: Visual Odometry with Minimal Assumptionscited by 4
  26. 2023Contrastive Learning Based Recursive Dynamic Multi-Scale Network for Image Derainingcited by 2
  27. 2023Generation of Radiology Findings in Chest X-Ray by Leveraging Collaborative Knowledgecited by 2
  28. 2024Distinctive Image Captioning: Leveraging Ground Truth Captions in CLIP Guided Reinforcement Learningcited by 2
  29. 2022ViFiCon: Vision and Wireless Association Via Self-Supervised Contrastive Learningcited by 1
  30. 2022How to Describe Images in a More Funny Way? Towards a Modular Approach to Cross-Modal Sarcasm Generationcited by 1
  31. 2023Transform, Contrast and Tell: Coherent Entity-Aware Multi-Image Captioningcited by 1
  32. 2021Towards Explainable Multi-Party Learning: A Contrastive Knowledge Sharing Frameworkcited by 0
  33. 2023Capturing Fine-grained Semantics in Contrastive Graph Representation Learningcited by 0
  34. 2023Contrastive Label Enhancementcited by 0
  35. 2024GRITv2: Efficient and Light-weight Social Relation Recognitioncited by 0
  36. 2024IVAC-P2L: Leveraging Irregular Repetition Priors for Improving Video Action Countingcited by 0
  37. 2024C3L: Content Correlated Vision-Language Instruction Tuning Data Generation via Contrastive Learningcited by 0
  38. 2024Surveying the Landscape of Image Captioning Evaluation: A Comprehensive Taxonomy, Trends and Metrics Analysiscited by 0

50 most cited of 54 All 54 (JSON)