# Recommending Themes for Ad Creative Design via Visual-Linguistic Representations

arXiv: https://arxiv.org/abs/2001.07194 · HTML: https://mlsanity.com/paper/2001.07194/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jan 2020.

## Built on by (most cited first, 15 of 15)

- [Visual Question Answering: which investigated applications?](https://mlsanity.com/paper/2103.02937/) (2021)
- [Staging E-Commerce Products for Online Advertising using Retrieval Assisted Image Generation](https://mlsanity.com/paper/2307.15326/) (2023)
- [TSI: an Ad Text Strength Indicator using Text-to-CTR and Semantic-Ad-Similarity](https://mlsanity.com/paper/2108.08226/) (2021)
- [PiggyBack: Pretrained Visual Question Answering Environment for Backing up Non-deep Learning Professionals](https://mlsanity.com/paper/2211.15940/) (2022)
- [SINGA-Easy: An Easy-to-Use Framework for MultiModal Analysis](https://mlsanity.com/paper/2108.02572/) (2021)
- [VisualTextRank: Unsupervised Graph-based Content Extraction for Automating Ad Text to Image Search](https://mlsanity.com/paper/2108.02725/) (2021)
- [Text is NOT Enough: Integrating Visual Impressions into Open-domain Dialogue Generation](https://mlsanity.com/paper/2109.05778/) (2021)
- [Discovering Personalized Semantics for Soft Attributes in Recommender Systems using Concept Activation Vectors](https://mlsanity.com/paper/2202.02830/) (2022)
- [Reinforced MOOCs Concept Recommendation in Heterogeneous Information Networks](https://mlsanity.com/paper/2203.11011/) (2022)
- [Explanation-Guided Fairness Testing through Genetic Algorithm](https://mlsanity.com/paper/2205.08335/) (2022)
- [Using Graph Neural Networks for Program Termination](https://mlsanity.com/paper/2207.14648/) (2022)
- [Tribe or Not? Critical Inspection of Group Differences Using TribalGram](https://mlsanity.com/paper/2303.09664/) (2023)
- [Improving Prediction Performance and Model Interpretability through Attention Mechanisms from Basic and Applied Research Perspectives](https://mlsanity.com/paper/2303.14116/) (2023)
- [Profiling the news spreading barriers using news headlines](https://mlsanity.com/paper/2304.11088/) (2023)
- [PinDCO: Whole-Page Aware Dynamic Creative Optimization at Scale](https://mlsanity.com/paper/2609.11943/) (2026)

## Abstract (arXiv)

There is a perennial need in the online advertising industry to refresh ad creatives, i.e., images and text used for enticing online users towards a brand. Such refreshes are required to reduce the likelihood of ad fatigue among online users, and to incorporate insights from other successful campaigns in related product categories. Given a brand, to come up with themes for a new ad is a painstaking and time consuming process for creative strategists. Strategists typically draw inspiration from the images and text used for past ad campaigns, as well as world knowledge on the brands. To automatically infer ad themes via such multimodal sources of information in past ad campaigns, we propose a theme (keyphrase) recommender system for ad creative strategists. The theme recommender is based on aggregating results from a visual question answering (VQA) task, which ingests the following: (i) ad images, (ii) text associated with the ads as well as Wikipedia pages on the brands in the ads, and (iii) questions around the ad. We leverage transformer based cross-modality encoders to train visual-linguistic representations for our VQA task. We study two formulations for the VQA task along the lines of classification and ranking; via experiments on a public dataset, we show that cross-modal representations lead to significantly better classification accuracy and ranking precision-recall metrics. Cross-modal representations show better performance compared to separate image and text representations. In addition, the use of multimodal information shows a significant lift over using only textual or visual information.