mlsanityData to 14 Oct 2026

Detailed Object Description with Controllable Dimensions

arXiv 2411.19106 ↗First posted 28 Nov 2024cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

Object description plays an important role for visually impaired individuals to understand and compare the differences between objects. Recent multimodal large language models(MLLMs) exhibit powerful perceptual abilities and demonstrate impressive potential for generating object-centric descriptions. However, the descriptions generated by such models may still usually contain a lot of content that is not relevant to the user intent or miss some important object dimension details. Under special scenarios, users may only need the details of certain dimensions of an object. In this paper, we propose a training-free object description refinement pipeline, Dimension Tailor, designed to enhance user-specified details in object descriptions. This pipeline includes three steps: dimension extracting, erasing, and supplementing, which decompose the description into user-specified dimensions. Dimension Tailor can not only improve the quality of object details but also offer flexibility in including or excluding specific dimensions based on user preferences. We conducted extensive experiments to demonstrate the effectiveness of Dimension Tailor on controllable object descriptions. Notably, the proposed pipeline can consistently improve the performance of the recent MLLMs. The code is currently accessible at https://github.com/xin-ran-w/ControllableObjectDescription.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 52 of its 81 references that are in the map. First posted on arXiv 28 Nov 2024.

Builds on

52 of 81 references.

References · 52

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2023Visual Instruction Tuningcited by 7,869
  5. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  6. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  7. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  8. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  9. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  10. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  11. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  12. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,423
38 more
  1. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,005
  2. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 940
  3. 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 824
  4. 2022When and why vision-language models behave like bags-of-words, and what to do about it?cited by 610
  5. 2023Ferret: Refer and Ground Anything Anywhere at Any Granularitycited by 509
  6. 2023TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answeringcited by 428
  7. 2023Recognize Anything: A Strong Image Tagging Modelcited by 309
  8. 2023GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interestcited by 302
  9. 2023Woodpecker: Hallucination Correction for Multimodal Large Language Modelscited by 226
  10. 2024LLaVA-UHD: an LMM Perceiving Any Aspect Ratio and High-Resolution Imagescited by 195
  11. 2015DenseCap: Fully Convolutional Localization Networks for Dense Captioningcited by 176
  12. 2023Davidsonian Scene Graph: Improving Reliability in Fine-grained Evaluation for Text-to-Image Generationcited by 171
  13. 2023Osprey: Pixel Understanding with Visual Instruction Tuningcited by 159
  14. 2023Alpha-CLIP: A CLIP Model Focusing on Wherever You Wantcited by 151
  15. 2024SEED-Bench-2-Plus: Benchmarking Multimodal Large Language Models with Text-Rich Visual Comprehensioncited by 146
  16. 2024DOCCI: Descriptions of Connected and Contrasting Imagescited by 132
  17. 2024OMG-LLaVA: Bridging Image-level, Object-level, Pixel-level Reasoning and Understandingcited by 127
  18. 2023Caption Anything: Interactive Image Description with Diverse Multimodal Controlscited by 99
  19. 2023SEED-Bench-2: Benchmarking Multimodal Large Language Modelscited by 97
  20. 2021Learning to Predict Visual Attributes in the Wildcited by 96
  21. 2020Say As You Wish: Fine-grained Control of Image Caption Generation with Abstract Scene Graphscited by 87
  22. 2023Tag2Text: Guiding Vision-Language Model via Image Taggingcited by 86
  23. 2023Dense and Aligned Captions (DAC) Promote Compositional Reasoning in VL Modelscited by 72
  24. 2023Open-Set Image Tagging with Multi-Grained Text Supervisioncited by 67
  25. 2018Show, Control and Tell: A Framework for Generating Controllable and Grounded Captionscited by 65
  26. 2024ImageInWords: Unlocking Hyper-Detailed Image Descriptionscited by 57
  27. 2015SentiCap: Generating Image Descriptions with Sentimentscited by 48
  28. 2019Describing like humans: on diversity in image captioningcited by 46
  29. 2023FaithScore: Fine-grained Evaluations of Hallucinations in Large Vision-Language Modelscited by 46
  30. 2023OvarNet: Towards Open-vocabulary Object Attribute Recognitioncited by 43
  31. 2021Human-like Controllable Image Captioning with Verb-specific Semantic Rolescited by 41
  32. 2020Length-Controllable Image Captioningcited by 36
  33. 2019Reflective Decoding Network for Image Captioningcited by 31
  34. 2021Boosting Entity-aware Image Captioning with Multi-modal Knowledge Graphcited by 31
  35. 2022Open-vocabulary Attribute Detectioncited by 29
  36. 2023Prompt Highlighter: Interactive Control for Multi-Modal LLMscited by 29
  37. 2020Fine-Grained Image Captioning with Global-Local Discriminative Objectivecited by 12
  38. 2024FlexCap: Describe Anything in Images in Controllable Detailcited by 7

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025Harnessing Caption Detailness for Data-Efficient Text-to-Image Generationcited by 4
  2. 2026From Competition to Coopetition: Coopetitive Training-Free Image Editing Based on Text Guidancecited by 1
  3. 2026RailGen: Improving Railway Intrusion Detection via Agent-Guided Small-Scale Foreign Object Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Guiding Multimodal Large Language Models with Blind and Low Vision People Visual Questions for Proactive Visual Interpretations
  2. 2025How Blind and Low-Vision Individuals Prefer Large Vision-Language Model-Generated Scene Descriptions
  3. 2026How Multimodal Large Language Models Support Access to Visual Information: A Diary Study With Blind and Low Vision People
  4. 2024Exploring Perceptual Limitation of Multimodal Large Language Models
  5. 2025Evaluating Multimodal Language Models as Visual Assistants for Visually Impaired Users
  6. 2024Reminding Multimodal Large Language Models of Object-aware Knowledge with Retrieved Tags
  7. 2023Synthesize, Diagnose, and Optimize: Towards Fine-Grained Vision-Language Understanding
  8. 2023VCoder: Versatile Vision Encoders for Multimodal Large Language Models