mlsanityData to 7 Oct 2026

Style-Aware Contrastive Learning for Multi-Style Image Captioning

arXiv 2301.11367 ↗First posted 26 Jan 2023cs.CV cs.CLcited by 21

Not on the map. Why?

Abstract (arXiv)

Existing multi-style image captioning methods show promising results in generating a caption with accurate visual content and desired linguistic style. However, existing methods overlook the relationship between linguistic style and visual content. To overcome this drawback, we propose style-aware contrastive learning for multi-style image captioning. First, we present a style-aware visual encoder with contrastive learning to mine potential visual content relevant to style. Moreover, we propose a style-aware triplet contrast objective to distinguish whether the image, style and caption matched. To provide positive and negative samples for contrastive learning, we present three retrieval schemes: object-based retrieval, RoI-based retrieval and triplet-based retrieval, and design a dynamic trade-off function to calculate retrieval scores. Experimental results demonstrate that our approach achieves state-of-the-art performance. In addition, we conduct an extensive analysis to verify the effectiveness of our method.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 27 of its 42 references that are in the map. First posted on arXiv 26 Jan 2023.

Builds on

27 of 42 references.

References · 27

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  6. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  7. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  8. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
  9. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,421
  10. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,310
  11. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,224
  12. 2014Show and Tell: A Neural Image Caption Generatorcited by 877
15 more
  1. 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 820
  2. 2018Exploring the Limits of Weakly Supervised Pretrainingcited by 604
  3. 2016Self-critical Sequence Training for Image Captioningcited by 544
  4. 2016Visual Dialogcited by 377
  5. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 304
  6. 2022Vision-Language Pre-Training with Triple Contrastive Learningcited by 212
  7. 2021Optimizing Dense Retrieval Model Training with Hard Negativescited by 154
  8. 2018Engaging Image Captioning Via Personalitycited by 51
  9. 2015SentiCap: Generating Image Descriptions with Sentimentscited by 48
  10. 2022ClarET: Pre-training a Correlation-Aware Context-To-Event Transformer for Event-Centric Generation and Classificationcited by 45
  11. 2018SemStyle: Learning to Generate Stylised Image Captions using Unaligned Textcited by 27
  12. 2018"Factual" or "Emotional": Stylized Image Captioning with Adaptive Learning and Attentioncited by 26
  13. 2021Similar Scenes arouse Similar Emotions: Parallel Data Augmentation for Stylized Image Captioningcited by 7
  14. 2020Structural and Functional Decomposition for Personality Image Captioning in a Communication Gamecited by 2
  15. 20213M: Multi-style image caption generation using Multi-modality features under Multi-UPDOWN modelcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 9; 2025: 10; 2026: 1.

21 papers.

Sort:

By area:

  1. 2023Retrieving Multimodal Information for Augmented Generation: A Surveycited by 84
  2. 2025Retrieval Augmented Generation and Understanding in Vision: A Survey and New Outlookcited by 32
  3. 2024MemoryMamba: Memory-Augmented State Space Model for Defect Recognitioncited by 24
  4. 2024Understanding Retrieval Robustness for Retrieval-Augmented Image Captioningcited by 9
  5. 2024Exploiting Diffusion Prior for Out-of-Distribution Detectioncited by 7
  6. 2025An LLM-LVLM Driven Agent for Iterative and Fine-Grained Image Editingcited by 6
  7. 2025Large Language Models for Zero-Shot Multicultural Name Recognitioncited by 1
  8. 2025LumiGen: An LVLM-Enhanced Iterative Framework for Fine-Grained Text-to-Image Generationcited by 1
  9. 2025XDR-LVLM: An Explainable Vision-Language Large Model for Diabetic Retinopathy Diagnosiscited by 1
  10. 2025PoultryTalk: A Multi-modal Retrieval-Augmented Generation (RAG) System for Intelligent Poultry Management and Decision Supportcited by 1
  11. 2024Surveying the Landscape of Image Captioning Evaluation: A Comprehensive Taxonomy, Trends and Metrics Analysiscited by 0
  12. 2024Exploring Large Vision-Language Models for Robust and Efficient Industrial Anomaly Detectioncited by 0
9 more
  1. 2024Reasoning-Aware Query-Focused Summarization over Multi-Table Datacited by 0
  2. 2024Bridging Vision and Language: Modeling Causality and Temporality in Video Narrativescited by 0
  3. 2024Optimizing Vision-Language Interactions Through Decoder-Only Modelscited by 0
  4. 2024Leveraging Retrieval-Augmented Tags for Large Vision-Language Understanding in Complex Scenescited by 0
  5. 2025Hierarchical Vision-Language Alignment for Text-to-Image Generation via Diffusion Modelscited by 0
  6. 2025Weak Supervision Dynamic KL-Weighted Diffusion Models Guided by Large Language Modelscited by 0
  7. 2025A Generative Framework for Bidirectional Image-Report Understanding in Chest Radiographycited by 0
  8. 2025Revolutionizing Radiology Workflow with Factual and Efficient CXR Report Generationcited by 0
  9. 2026When Does External Guidance Help LLM Reasoning? A Bias-Variance Theory of Guidance-Augmented GRPOcited by 0