ImageSet2Text: Describing Sets of Images through Text
Not on the map. Why?
Abstract (arXiv)
In the era of large-scale visual data, understanding collections of images is a challenging yet important task. To this end, we introduce ImageSet2Text, a novel method to automatically generate natural language descriptions of image sets. Based on large language models, visual-question answering chains, an external lexical graph, and CLIP-based verification, ImageSet2Text iteratively extracts key concepts from image subsets and organizes them into a structured concept graph. We conduct extensive experiments evaluating the quality of the generated descriptions in terms of accuracy, completeness, and user satisfaction. We also examine the method's behavior through ablation studies, scalability assessments, and failure analyses. Results demonstrate that ImageSet2Text combines data-driven AI and symbolic representations to reliably summarize large image collections for a wide range of applications.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 43 of its 70 references that are in the map. First posted on arXiv 25 Mar 2025.
Builds on
43 of 70 references.
References · 43
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2023GPT-4 Technical Reportcited by 18,396
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2023Visual Instruction Tuningcited by 7,869
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
- 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
- 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
- 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
31 more
- 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,046
- 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,423
- 2018Datasheets for Datasetscited by 1,386
- 2020Concept Bottleneck Modelscited by 1,032
- 2014Show and Tell: A Neural Image Caption Generatorcited by 879
- 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 824
- 2023TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answeringcited by 428
- 2023TRAK: Attributing Model Behavior at Scalecited by 283
- 2023MIMIC-IT: Multi-Modal In-Context Instruction Tuningcited by 267
- 2018Deep Learning: A Critical Appraisalcited by 217
- 2018A Comprehensive Survey of Deep Learning for Image Captioningcited by 214
- 2022Domino: Discovering Systematic Errors with Cross-Modal Embeddingscited by 156
- 2020REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasetscited by 136
- 2020Captioning Images Taken by People Who Are Blindcited by 110
- 2024MMDU: A Multi-Turn Multi-Image Dialog Understanding Benchmark and Instruction-Tuning Dataset for LVLMscited by 90
- 2023ChatGPT Asks, BLIP-2 Answers: Automatic Questioning Towards Enriched Visual Descriptionscited by 86
- 2024MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Modelscited by 67
- 2024OpenBias: Open-set Bias Detection in Text-to-Image Generative Modelscited by 61
- 2023Describing Differences in Image Sets with Natural Languagecited by 53
- 2023Video ChatCaptioner: Towards Enriched Spatiotemporal Descriptionscited by 42
- 2023Changes to Captions: An Attentive Network for Remote Sensing Change Captioningcited by 41
- 2022Image Difference Captioning with Pre-training and Contrastive Learningcited by 39
- 2021Explain Me the Painting: Multi-Topic Knowledgeable Art Description Generationcited by 30
- 2022ModelDiff: A Framework for Comparing Learning Algorithmscited by 30
- 2023Data-centric Graph Learning: A Surveycited by 22
- 2024Explain via Any Concept: Concept Bottleneck Model with Open Vocabulary Conceptscited by 20
- 2020Context-Aware Group Captioning via Self-Attention and Contrastive Featurescited by 17
- 2024Concept Bottleneck Models Without Predefined Conceptscited by 17
- 2023Prototype-based Dataset Comparisoncited by 9
- 2019Hierarchical Photo-Scene Encoder for Album Storytellingcited by 5
- 2024Controllable Contextualized Image Captioning: Directing the Visual Narrative through User-Defined Highlightscited by 3
Built on by
Not cited yet.