mlsanityData to 7 Oct 2026

KAFA: Rethinking Image Ad Understanding with Knowledge-Augmented Feature Adaptation of Vision-Language Models

arXiv 2305.18373 ↗First posted 28 May 2023cs.CV cs.CLcited by 7

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Image ad understanding is a crucial task with wide real-world applications. Although highly challenging with the involvement of diverse atypical scenes, real-world entities, and reasoning over scene-texts, how to interpret image ads is relatively under-explored, especially in the era of foundational vision-language models (VLMs) featuring impressive generalizability and adaptability. In this paper, we perform the first empirical study of image ad understanding through the lens of pre-trained VLMs. We benchmark and reveal practical challenges in adapting these VLMs to image ad understanding. We propose a simple feature adaptation strategy to effectively fuse multimodal information for image ads and further empower it with knowledge of real-world entities. We hope our study draws more attention to image ad understanding which is broadly relevant to the advertising industry.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 20 of its 53 references that are in the map. First posted on arXiv 28 May 2023.

Builds on

20 of 53 references.

References · 20

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  3. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  4. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  5. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  6. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,265
  7. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapterscited by 1,021
  8. 2021Florence: A New Foundation Model for Computer Visioncited by 802
  9. 2021Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modelingcited by 429
  10. 2021How Much Can CLIP Benefit Vision-and-Language Tasks?cited by 321
  11. 2021CPT: Colorful Prompt Tuning for Pre-trained Vision-Language Modelscited by 182
  12. 2021Combined Scaling for Zero-shot Transfer Learningcited by 182
8 more
  1. 2021KAT: A Knowledge Augmented Transformer for Vision-and-Languagecited by 125
  2. 2022CLIP Models are Few-shot Learners: Empirical Studies on VQA and Visual Entailmentcited by 113
  3. 2015Explicit Knowledge-based Reasoning for Visual Question Answeringcited by 85
  4. 2022SVL-Adapter: Self-Supervised Adapter for Vision-Language Pretrained Modelscited by 34
  5. 2022MetaCLUE: Towards Comprehensive Visual Metaphors Researchcited by 28
  6. 2022Tiny-Attention Adapter: Contexts Are More Important Than the Number of Parameterscited by 12
  7. 2022Ask Me What You Need: Product Retrieval using Knowledge from GPT-3cited by 8
  8. 2018Scalable Deep Learning Logo Detectioncited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 3; 2026: 2.

7 papers.

  1. 2025AdsQA: Towards Advertisement Video Understandingcited by 12
  2. 2024Benchmarking VLMs' Reasoning About Persuasive Atypical Imagescited by 5
  3. 2024Don't Buy it! Reassessing the Ad Understanding Abilities of Contrastive Multimodal Modelscited by 2
  4. 2025MARS2 2025 Challenge on Multimodal Reasoning: Datasets, Methods, Results, Discussion, and Outlookcited by 1
  5. 2025Augmented Vision-Language Models: A Systematic Reviewcited by 0
  6. 2026A3: Towards Advertising Aesthetic Assessmentcited by 0
  7. 2026Computational Humor with Multimodal LLMs: Methods, Datasets, Evaluation, and Challengescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026MLLM-VADStory: Domain Knowledge-Driven Multimodal LLMs for Video Ad Storyline Insights
  2. 2025On the Limitations of Vision-Language Models in Understanding Image Transforms
  3. 2024Exploring the Frontier of Vision-Language Models: A Survey of Current Methodologies and Future Directions
  4. 2026Adapting Vision-Language Models for E-commerce Understanding at Scale
  5. 2024The Narrow Gate: Localized Image-Text Communication in Native Multimodal Models
  6. 2023Enhancing Dynamic Image Advertising with Vision-Language Pre-training
  7. 2024What's in the Image? A Deep-Dive into the Vision of Vision Language Models
  8. 2024Vision Search Assistant: Empower Vision-Language Models as Multimodal Search Engines