mlsanityData to 7 Oct 2026

Exploring the Distinctiveness and Fidelity of the Descriptions Generated by Large Vision-Language Models

arXiv 2404.17534 ↗First posted 26 Apr 2024cs.CV cs.MMcited by 1

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Large Vision-Language Models (LVLMs) are gaining traction for their remarkable ability to process and integrate visual and textual data. Despite their popularity, the capacity of LVLMs to generate precise, fine-grained textual descriptions has not been fully explored. This study addresses this gap by focusing on textitdistinctiveness and textitfidelity, assessing how models like Open-Flamingo, IDEFICS, and MiniGPT-4 can distinguish between similar objects and accurately describe visual features. We proposed the Textual Retrieval-Augmented Classification (TRAC) framework, which, by leveraging its generative capabilities, allows us to delve deeper into analyzing fine-grained visual description generation. This research provides valuable insights into the generation quality of LVLMs, enhancing the understanding of multimodal language models. Notably, MiniGPT-4 stands out for its better ability to generate fine-grained descriptions, outperforming the other two models in this aspect. The code is provided at urlhttps://anonymous.4open.science/r/ExploreFGVDs-E277.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 23 of its 41 references that are in the map. First posted on arXiv 26 Apr 2024.

Builds on

23 of 41 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Vision-Language Models for Vision Tasks: A Survey3 Apr
  2. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation28 Jan

Other references · 21

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  5. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  6. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
  7. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
  8. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,352
  9. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,310
  10. 2014Show and Tell: A Neural Image Caption Generatorcited by 877
  11. 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 820
  12. 2021ClipCap: CLIP Prefix for Image Captioningcited by 557
9 more
  1. 2022Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learningcited by 553
  2. 2019Image Captioning: Transforming Objects into Wordscited by 171
  3. 2020Say As You Wish: Fine-grained Control of Image Caption Generation with Abstract Scene Graphscited by 87
  4. 2022Fine-grained Image Captioning with CLIP Rewardcited by 69
  5. 2023Exploring Diverse In-Context Configurations for Image Captioningcited by 63
  6. 2023Enhancing CLIP with GPT-4: Harnessing Visual Descriptions as Promptscited by 48
  7. 2023De-Diffusion Makes Text a Strong Cross-Modal Interfacecited by 14
  8. 2023Multimodal Foundation Models for Zero-shot Animal Species Recognition in Camera Trap Imagescited by 13
  9. 2023Manipulating the Label Space for In-Context Classificationcited by 5

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024Pseudo-RIS: Distinctive Pseudo-supervision Generation for Referring Image Segmentationcited by 13

Similar, not cited

Close in wording, no citation either way

  1. 2023Synthesize, Diagnose, and Optimize: Towards Fine-Grained Vision-Language Understanding
  2. 2025Beyond Accuracy: Metrics that Uncover What Makes a 'Good' Visual Descriptor
  3. 2024Finer: Investigating and Enhancing Fine-Grained Visual Concept Recognition in Large Vision Language Models
  4. 2025Growing Visual Generative Capacity for Pre-Trained MLLMs
  5. 2024Exploring the Frontier of Vision-Language Models: A Survey of Current Methodologies and Future Directions
  6. 2025Benchmarking Large Vision-Language Models on Fine-Grained Image Tasks: A Comprehensive Evaluation
  7. 2025Same or Not? Enhancing Visual Perception in Vision-Language Models
  8. 2023Revisiting the Role of Language Priors in Vision-Language Models