mlsanityData to 7 Oct 2026

VLRM: Vision-Language Models act as Reward Models for Image Captioning

arXiv 2404.01911 ↗First posted 2 Apr 2024cs.CVcited by 18

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

also in 1 more
Abstract (arXiv)

In this work, we present an unsupervised method for enhancing an image captioning model (in our case, BLIP2) using reinforcement learning and vision-language models like CLIP and BLIP2-ITM as reward models. The RL-tuned model is able to generate longer and more comprehensive descriptions. Our model reaches impressive 0.90 R@1 CLIP Recall score on MS-COCO Carpathy Test Split. Weights are available at https://huggingface.co/sashakunitsyn/vlrm-blip2-opt-2.7b.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 21 of its 26 references that are in the map. First posted on arXiv 2 Apr 2024.

Builds on

21 of 26 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022Flamingo: a Visual Language Model for Few-Shot Learning29 Apr
  2. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation28 Jan

Other references · 19

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2017Proximal Policy Optimization Algorithmscited by 14,441
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  8. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  9. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  10. 2017Deep reinforcement learning from human preferencescited by 3,846
  11. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  12. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
7 more
  1. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  2. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,265
  3. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,209
  4. 2022Socratic Models: Composing Zero-Shot Multimodal Reasoning with Languagecited by 513
  5. 2023Image Captioners Are Scalable Vision Learners Toocited by 89
  6. 2023ChatGPT Asks, BLIP-2 Answers: Automatic Questioning Towards Enriched Visual Descriptionscited by 86
  7. 2023IC3: Image Captioning by Committee Consensuscited by 20

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 7; 2026: 10.

18 papers.

Sort:

By area:

  1. 2024EmoLLM: Multimodal Emotional Understanding Meets Large Language Modelscited by 31
  2. 2025Why Reasoning Matters? A Survey of Advancements in Multimodal Reasoning (v1)cited by 23
  3. 2025QSVD: Efficient Low-rank Approximation for Unified Query-Key-Value Weight Compression in Low-Precision Vision-Language Modelscited by 15
  4. 2025Adaptive Divergence Regularized Policy Optimization for Fine-tuning Generative Modelscited by 6
  5. 2026WSVD: Weighted Low-Rank Approximation for Fast and Efficient Execution of Low-Precision Vision-Language Modelscited by 3
  6. 2025ZIUM: Zero-Shot Intent-Aware Adversarial Attack on Unlearned Modelscited by 1
  7. 2025What Makes a Good Generated Image? Investigating Human and Multimodal LLM Image Preference Alignmentcited by 1
  8. 2025Clarification as Supervision: Reinforcement Learning for Vision-Language Interfacescited by 1
  9. 2025Reasoning Like Experts: Leveraging Multimodal Large Language Models for Drawing-based Psychoanalysiscited by 1
  10. 2026Multimodal Backdoor Attack on VLMs for Autonomous Driving via Graffiti and Cross-Lingual Triggerscited by 1
  11. 2026DREAM-S: Speculative Decoding with Searchable Drafting and Target-Aware Refinement for Multimodal Generationcited by 1
  12. 2026FigEx2: Visual-Conditioned Panel Detection and Captioning for Scientific Compound Figurescited by 0
6 more
  1. 2026Grandes Modelos de Linguagem Multimodais (MLLMs): Da Teoria `a Pr'aticacited by 0
  2. 2026Cross-modal Identity Mapping: Minimizing Information Loss in Modality Conversion via Reinforcement Learningcited by 0
  3. 2026Geometry-Aware Semantic Reasoning for Training Free Video Anomaly Detectioncited by 0
  4. 2026LASER: Loss-Aware Singular-value Decomposition and Rank Allocation for Efficient Low-Precision Vision-Language Modelscited by 0
  5. 2026Re3Cap: Retrieval-Guided Refinement for Image Captioning Enhancement via Reinforcement Learningcited by 0
  6. 2026LineupRL: Verifiable Reinforcement Learning for Time Series Captioning via Caption-to-Series Identificationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learning
  2. 2026CapRL++: Unified Reinforcement Learning with Verifiable Rewards for Dense Image and Video Captioning
  3. 2026BalCapRL: A Balanced Framework for RL-Based MLLM Image Captioning
  4. 2023FuseCap: Leveraging Large Language Models for Enriched Fused Image Captions
  5. 2024Distinctive Image Captioning: Leveraging Ground Truth Captions in CLIP Guided Reinforcement Learning
  6. 2023Test-Time Adaptation with CLIP Reward for Zero-Shot Generalization in Vision-Language Models
  7. 2026PromptEcho: Annotation-Free Reward from Vision-Language Models for Text-to-Image Reinforcement Learning
  8. 2025From Captions to Rewards (CAREVL): Leveraging Large Language Model Experts for Enhanced Reward Modeling in Large Vision-Language Models