mlsanityData to 14 Oct 2026

Enhancing Dynamic Image Advertising with Vision-Language Pre-training

arXiv 2306.14112 ↗First posted 25 Jun 2023cs.IRcited by 2

On the map: Multimodal LLMs / vision-language models → Document understanding and multimodal RAG

Abstract (arXiv)

In the multimedia era, image is an effective medium in search advertising. Dynamic Image Advertising (DIA), a system that matches queries with ad images and generates multimodal ads, is introduced to improve user experience and ad revenue. The core of DIA is a query-image matching module performing ad image retrieval and relevance modeling. Current query-image matching suffers from limited and inconsistent data, and insufficient cross-modal interaction. Also, the separate optimization of retrieval and relevance models affects overall performance. To address this issue, we propose a vision-language framework consisting of two parts. First, we train a base model on large-scale image-text pairs to learn general multimodal representation. Then, we fine-tune the base model on advertising business data, unifying relevance modeling and retrieval through multi-objective learning. Our framework has been implemented in Baidu search advertising system "Phoneix Nest". Online evaluation shows that it improves cost per mille (CPM) and click-through rate (CTR) by 1.04% and 1.865%.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 28 of its 36 references that are in the map. First posted on arXiv 25 Jun 2023.

Builds on

28 of 36 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021Kaleido-BERT: Vision-Language Pre-training on Fashion Domain30 Mar

Other references · 27

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  7. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  8. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  9. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  10. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,269
  11. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,619
  12. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,274
15 more
  1. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,230
  2. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,214
  3. 2021SimVLM: Simple Visual Language Model Pretraining with Weak Supervisioncited by 647
  4. 2021VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Expertscited by 425
  5. 2022Wukong: A 100 Million Large-scale Chinese Cross-modal Pre-training Benchmarkcited by 114
  6. 2022Chinese CLIP: Contrastive Vision-Language Pretraining in Chinesecited by 109
  7. 2018COCO-CN for Cross-Lingual Image Tagging, Captioning and Retrievalcited by 80
  8. 2020FashionBERT: Text and Image Matching with Adaptive Loss for Cross-modal Retrievalcited by 55
  9. 2021TextGNN: Improving Text Encoder via Graph Neural Network in Sponsored Searchcited by 54
  10. 2021AdsGNN: Behavior-Graph Augmented Relevance Modeling in Sponsored Searchcited by 38
  11. 2017Fluency-Guided Cross-Lingual Image Captioningcited by 33
  12. 2022Uni-Retriever: Towards Learning The Unified Embedding Based Retriever in Bing Sponsored Searchcited by 17
  13. 2022ERNIE-ViL 2.0: Multi-view Contrastive Learning for Image-Text Pre-trainingcited by 8
  14. 2022Boost CTR Prediction for New Advertisements via Modeling Visual Contentcited by 5
  15. 2022AMCAD: Adaptive Mixed-Curvature Representation based Advertisement Retrieval Systemcited by 4

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025SUMMA: A Multimodal Large Language Model for Advertisement Summarizationcited by 2
  2. 2025PATFinger: Prompt-Adapted Transferable Fingerprinting against Unauthorized Multimodal Dataset Usagecited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025HCMRM: A High-Consistency Multimodal Relevance Model for Search Ads
  2. 2025CTR-Driven Advertising Image Generation with Multimodal Large Language Models
  3. 2023KAFA: Rethinking Image Ad Understanding with Knowledge-Augmented Feature Adaptation of Vision-Language Models
  4. 2024ContextIQ: A Multimodal Expert-Based Video Retrieval System for Contextual Advertising
  5. 2023MAKE: Vision-Language Pre-training based Product Retrieval in Taobao Search
  6. 2024Don't Buy it! Reassessing the Ad Understanding Abilities of Contrastive Multimodal Models
  7. 2024Bringing Multimodality to Amazon Visual Search System
  8. 2023Query-LIFE: Query-aware Language Image Fusion Embedding for E-Commerce Relevance