mlsanityData to 7 Oct 2026

CCCaption: Dual-Reward Reinforcement Learning for Complete and Correct Image Captioning

arXiv 2602.21655 ↗First posted 25 Feb 2026cs.CV cs.AIcited by 7

Not on the map. Why?

Abstract (arXiv)

Image captioning remains a fundamental task for vision language understanding, yet ground-truth supervision still relies predominantly on human-annotated references. Because human annotations reflect subjective preferences and expertise, ground-truth captions are often incomplete or even incorrect, which in turn limits caption models. We argue that caption quality should be assessed by two objective aspects: completeness (does the caption cover all salient visual facts?) and correctness (are the descriptions true with respect to the image?). To this end, we introduce CCCaption: a dual-reward reinforcement learning framework with a dedicated fine-tuning corpus that explicitly optimizes these properties to generate Complete and Correct Captions. For completeness, we use diverse LVLMs to disentangle the image into a set of visual queries, and reward captions that answer more of these queries, with a dynamic query sampling strategy to improve training efficiency. For correctness, we penalize captions that contain hallucinations by validating the authenticity of sub-caption queries, which are derived from the caption decomposition. Our symmetric dual-reward optimization jointly maximizes completeness and correctness, guiding models toward captions that better satisfy these objective criteria. Extensive experiments across standard captioning benchmarks show consistent improvements, offering a principled path to training caption models beyond human-annotation imitation.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 44 of its 59 references that are in the map. First posted on arXiv 25 Feb 2026.

Builds on

44 of 59 references.

References · 44

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  5. 2025Qwen3 Technical Reportcited by 8,195
  6. 2023Visual Instruction Tuningcited by 7,792
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  8. 2025Qwen2.5-VL Technical Reportcited by 5,418
  9. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  10. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  11. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  12. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
32 more
  1. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  2. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  3. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  4. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  5. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
  6. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,449
  7. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,421
  8. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  9. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,291
  10. 2023A Survey on Multimodal Large Language Modelscited by 898
  11. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 893
  12. 2014Show and Tell: A Neural Image Caption Generatorcited by 877
  13. 2016A Diagram Is Worth A Dozen Imagescited by 793
  14. 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 573
  15. 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Modelscited by 572
  16. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 494
  17. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 466
  18. 2021InfographicVQAcited by 430
  19. 2024CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMscited by 229
  20. 2019Image Captioning: Transforming Objects into Wordscited by 171
  21. 2023Improving Multimodal Datasets with Image Captioningcited by 93
  22. 2018Unsupervised Image Captioningcited by 79
  23. 2025CapArena: Benchmarking and Analyzing Detailed Image Captioning in the LLM Eracited by 40
  24. 2017Deep Reinforcement Learning-based Image Captioning with Embedding Rewardcited by 39
  25. 2025CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learningcited by 36
  26. 2024Prism: A Framework for Decoupling and Assessing the Capabilities of VLMscited by 32
  27. 2025We-Math 2.0: A Versatile MathBook System for Incentivizing Visual Mathematical Reasoningcited by 32
  28. 2024Personalizing Multimodal Large Language Models for Image Captioning: An Experimental Analysiscited by 23
  29. 2025Painting with Words: Elevating Detailed Image Captioning with Benchmark and Alignment Learningcited by 23
  30. 2024Toward Robust Hyper-Detailed Image Captioning: A Multiagent Approach and Dual Evaluation Metrics for Factuality and Coveragecited by 20
  31. 2024Revisit Large-Scale Image-Caption Data in Pre-training Multimodal Foundation Modelscited by 14
  32. 2024Do More Details Always Introduce More Hallucinations in LVLM-based Image Captioning?cited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 7.

7 papers.