mlsanityData to 14 Oct 2026

RePIC: Reinforced Post-Training for Personalizing Multi-Modal Language Models

arXiv 2506.18369 ↗First posted 23 Jun 2025cs.CVcited by 13

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Recent multi-modal large language models (MLLMs) often struggle to generate personalized image captions, even when trained on high-quality captions. In this work, we observe that such limitations persist in existing post-training-based MLLM personalization methods. Specifically, despite being post-tuned with large-scale caption data through supervised fine-tuning (SFT), these models frequently fail to produce faithful descriptions in real-world scenarios, such as multi-concept image captioning. However, acquiring large-scale, high-quality captions for such complex settings is both costly and difficult. To address the data-centric nature of SFT, we propose a reinforcement learning (RL)-based post-training framework. To the best of our knowledge, this is the first RL-based approach to post-train MLLMs for personalized image captioning. Our method significantly enhances both visual recognition and personalized generation capabilities of MLLMs, and consistently outperforms existing SFT-based baselines, especially in the challenging multi-concept image captioning task. Project page: https://github.com/oyt9306/RePIC

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 53 of its 66 references that are in the map. First posted on arXiv 23 Jun 2025.

Builds on

53 of 66 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Personalized Visual Instruction Tuning9 Oct
  2. 2024MyVLM: Personalizing VLMs for User-Specific Queries21 Mar

Other references · 51

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2017Proximal Policy Optimization Algorithmscited by 15,215
  5. 2024The Llama 3 Herd of Modelscited by 14,715
  6. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  7. 2025Qwen3 Technical Reportcited by 8,820
  8. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  9. 2023Visual Instruction Tuningcited by 7,869
  10. 2025Qwen2.5-VL Technical Reportcited by 5,587
  11. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  12. 2024GPT-4o System Cardcited by 4,588
38 more
  1. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  2. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  3. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  4. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  5. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  6. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  7. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  8. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,046
  9. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  10. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
  11. 2025s1: Simple test-time scalingcited by 1,341
  12. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,077
  13. 2016Modeling Context in Referring Expressionscited by 1,063
  14. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
  15. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 982
  16. 2024Gemma: Open Models Based on Gemini Research and Technologycited by 916
  17. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 904
  18. 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 824
  19. 2025Qwen2.5-Omni Technical Reportcited by 736
  20. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 672
  21. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 665
  22. 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 584
  23. 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Modelcited by 526
  24. 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 513
  25. 2024YOLO-World: Real-Time Open-Vocabulary Object Detectioncited by 503
  26. 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 491
  27. 2023RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedbackcited by 412
  28. 2024OminiControl: Minimal and Universal Control for Diffusion Transformercited by 340
  29. 2023How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Compositioncited by 223
  30. 2024DreamBench++: A Human-Aligned Benchmark for Personalized Image Generationcited by 131
  31. 2024Improving Diffusion Models for Authentic Virtual Try-on in the Wildcited by 109
  32. 2024Yo'LLaVA: Your Personalized Language and Vision Assistantcited by 62
  33. 2024UniPortrait: A Unified Framework for Identity-Preserving Single- and Multi-Human Image Personalizationcited by 47
  34. 2024Rethinking Data Selection for Supervised Fine-Tuningcited by 35
  35. 2024Identity Decoupling for Multi-Subject Personalization of Text-to-Image Modelscited by 33
  36. 2024InstantFamily: Masked Attention for Zero-shot Multi-ID Image Generationcited by 30
  37. 2024Multi-modal Preference Alignment Remedies Degradation of Visual Instruction Tuning on Language Modelscited by 26
  38. 2024Take the essence and discard the dross: A Rethinking on Data Selection for Fine-Tuning Large Language Modelscited by 12

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 12.

13 papers.

Sort:

By area:

  1. 2026PersonaVLM: Long-Term Personalized Multimodal LLMscited by 11
  2. 2025Reinforcement Learning for Large Model: A Surveycited by 9
  3. 2026PEARL: Personalized Streaming Video Understanding Modelcited by 9
  4. 2026Personal Visual Context Learning in Large Multimodal Modelscited by 3
  5. 2026Joint Continual Learning of Local Language Models and Cloud Offloading Decisions with Budget Constraintscited by 2
  6. 2026Contextualized Visual Personalization in Vision-Language Modelscited by 2
  7. 2026Omni-Persona: Systematic Benchmarking and Improving Omnimodal Personalizationcited by 1
  8. 2026Reasoning Beyond Literal: Cross-style Multimodal Reasoning for Figurative Language Understandingcited by 0
  9. 2026Culture-Aware Humorous Captioning: Multimodal Humor Generation across Cultural Contextscited by 0
  10. 2026Towards Customized Multimodal Role-Playcited by 0
  11. 2026Personalizing Embodied Multimodal Large Language Model Agents over Long-term User Interactionscited by 0
  12. 2026Personalizing MLLMs via Reinforced Multimodal Reference Gamecited by 0
1 more
  1. 2026Routing Without Training: Controllable-Ratio LLM Offloading via Reliability Gatingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learning
  2. 2026CapRL++: Unified Reinforcement Learning with Verifiable Rewards for Dense Image and Video Captioning
  3. 2025MM-R1: Unleashing the Power of Unified Multimodal Large Language Models for Personalized Image Generation
  4. 2026BalCapRL: A Balanced Framework for RL-Based MLLM Image Captioning
  5. 2024Learn from Downstream and Be Yourself in Multimodal Large Language Model Fine-Tuning
  6. 2025Keeping Yourself is Important in Downstream Tuning Multimodal Large Language Model
  7. 2024Personalizing Multimodal Large Language Models for Image Captioning: An Experimental Analysis
  8. 2026Re3Cap: Retrieval-Guided Refinement for Image Captioning Enhancement via Reinforcement Learning