mlsanityData to 14 Oct 2026

ImageSet2Text: Describing Sets of Images through Text

arXiv 2503.19361 ↗First posted 25 Mar 2025cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

In the era of large-scale visual data, understanding collections of images is a challenging yet important task. To this end, we introduce ImageSet2Text, a novel method to automatically generate natural language descriptions of image sets. Based on large language models, visual-question answering chains, an external lexical graph, and CLIP-based verification, ImageSet2Text iteratively extracts key concepts from image subsets and organizes them into a structured concept graph. We conduct extensive experiments evaluating the quality of the generated descriptions in terms of accuracy, completeness, and user satisfaction. We also examine the method's behavior through ablation studies, scalability assessments, and failure analyses. Results demonstrate that ImageSet2Text combines data-driven AI and symbolic representations to reliably summarize large image collections for a wide range of applications.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 43 of its 70 references that are in the map. First posted on arXiv 25 Mar 2025.

Builds on

43 of 70 references.

References · 43

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  5. 2023Visual Instruction Tuningcited by 7,869
  6. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  7. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  8. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  9. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  10. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  11. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  12. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
31 more
  1. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,046
  2. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,423
  3. 2018Datasheets for Datasetscited by 1,386
  4. 2020Concept Bottleneck Modelscited by 1,032
  5. 2014Show and Tell: A Neural Image Caption Generatorcited by 879
  6. 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 824
  7. 2023TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answeringcited by 428
  8. 2023TRAK: Attributing Model Behavior at Scalecited by 283
  9. 2023MIMIC-IT: Multi-Modal In-Context Instruction Tuningcited by 267
  10. 2018Deep Learning: A Critical Appraisalcited by 217
  11. 2018A Comprehensive Survey of Deep Learning for Image Captioningcited by 214
  12. 2022Domino: Discovering Systematic Errors with Cross-Modal Embeddingscited by 156
  13. 2020REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasetscited by 136
  14. 2020Captioning Images Taken by People Who Are Blindcited by 110
  15. 2024MMDU: A Multi-Turn Multi-Image Dialog Understanding Benchmark and Instruction-Tuning Dataset for LVLMscited by 90
  16. 2023ChatGPT Asks, BLIP-2 Answers: Automatic Questioning Towards Enriched Visual Descriptionscited by 86
  17. 2024MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Modelscited by 67
  18. 2024OpenBias: Open-set Bias Detection in Text-to-Image Generative Modelscited by 61
  19. 2023Describing Differences in Image Sets with Natural Languagecited by 53
  20. 2023Video ChatCaptioner: Towards Enriched Spatiotemporal Descriptionscited by 42
  21. 2023Changes to Captions: An Attentive Network for Remote Sensing Change Captioningcited by 41
  22. 2022Image Difference Captioning with Pre-training and Contrastive Learningcited by 39
  23. 2021Explain Me the Painting: Multi-Topic Knowledgeable Art Description Generationcited by 30
  24. 2022ModelDiff: A Framework for Comparing Learning Algorithmscited by 30
  25. 2023Data-centric Graph Learning: A Surveycited by 22
  26. 2024Explain via Any Concept: Concept Bottleneck Model with Open Vocabulary Conceptscited by 20
  27. 2020Context-Aware Group Captioning via Self-Attention and Contrastive Featurescited by 17
  28. 2024Concept Bottleneck Models Without Predefined Conceptscited by 17
  29. 2023Prototype-based Dataset Comparisoncited by 9
  30. 2019Hierarchical Photo-Scene Encoder for Album Storytellingcited by 5
  31. 2024Controllable Contextualized Image Captioning: Directing the Visual Narrative through User-Defined Highlightscited by 3

Built on by

Not cited yet.