RACap: Relation-Aware Prompting for Lightweight Retrieval-Augmented Image Captioning
On the map: Retrieval-augmented generation → Multimodal RAG
Abstract (arXiv)
Recent retrieval-augmented image captioning methods incorporate external knowledge to compensate for the limitations in comprehending complex scenes. However, current approaches face challenges in relation modeling: (1) the representation of semantic prompts is too coarse-grained to capture fine-grained relationships; (2) these methods lack explicit modeling of image objects and their semantic relationships. To address these limitations, we propose RACap, a relation-aware retrieval-augmented model for image captioning, which not only mines structured relation semantics from retrieval captions, but also identifies heterogeneous objects from the image. RACap effectively retrieves structured relation features that contain heterogeneous visual information to enhance the semantic consistency and relational expressiveness. Experimental results show that RACap, with only 10.8M trainable parameters, achieves superior performance compared to previous lightweight captioning models.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal RAG, one of 15 directions in Retrieval-augmented generation. Builds on 21 of its 25 references that are in the map. First posted on arXiv 19 Sep 2025.
Builds on
21 of 25 references.
Closest ancestors · 1
Same direction, newest first
Other references · 20
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
- 2016Gaussian Error Linear Units (GELUs)cited by 3,502
- 2017Billion-scale similarity search with GPUscited by 2,497
- 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,046
- 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,362
- 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,214
- 2014Deep Visual-Semantic Alignments for Generating Image Descriptionscited by 968
- 2020Object-Centric Learning with Slot Attentioncited by 876
- 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 824
8 more
- 2021ClipCap: CLIP Prefix for Image Captioningcited by 559
- 2018nocaps: novel object captioning at scalecited by 444
- 2022SmallCap: Lightweight Image Captioning Prompted with Retrieval Augmentationcited by 82
- 2024MeaCap: Memory-Augmented Zero-shot Image Captioningcited by 35
- 2023Transferable Decoding with Visual Entities for Zero-Shot Image Captioningcited by 34
- 2022Improving Cross-Modal Retrieval with Set of Diverse Embeddingscited by 33
- 2022I-Tuning: Tuning Frozen Language Models with Image for Lightweight Image Captioningcited by 16
- 2023RCA-NOC: Relative Contrastive Alignment for Novel Object Captioningcited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025DualCap: Enhancing Lightweight Image Captioning via Dual Retrieval with Similar Scenes Visual Prompts
- 2024Relation-aware Hierarchical Prompt for Open-vocabulary Scene Graph Generation
- 2024Towards Retrieval-Augmented Architectures for Image Captioning
- 2025RORPCap: Retrieval-based Objects and Relations Prompt for Image Captioning
- 2022PromptCap: Prompt-Guided Task-Aware Image Captioning
- 2024Understanding Retrieval Robustness for Retrieval-Augmented Image Captioning
- 2026Inference-Time Structural Reasoning for Compositional Vision-Language Understanding
- 2023Re-ViLM: Retrieval-Augmented Visual Language Model for Zero and Few-Shot Image Captioning