# Equal But Not The Same: Understanding the Implicit Relationship Between Persuasive Images and Text

arXiv: https://arxiv.org/abs/1807.08205 · HTML: https://mlsanity.com/paper/1807.08205/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jul 2018.

## Built on by (most cited first, 15 of 15)

- [MMoE: Enhancing Multimodal Models with Mixtures of Multimodal Interaction Experts](https://mlsanity.com/paper/2311.09580/) (2023)
- [Concadia: Towards Image-Based Text Generation with a Purpose](https://mlsanity.com/paper/2104.08376/) (2021)
- [ImageArg: A Multi-modal Tweet Dataset for Image Persuasiveness Mining](https://mlsanity.com/paper/2209.06416/) (2022)
- [Analyzing Online Political Advertisements](https://mlsanity.com/paper/2105.04047/) (2021)
- [A Video Is Worth 4096 Tokens: Verbalize Videos To Understand Them In Zero Shot](https://mlsanity.com/paper/2305.09758/) (2023)
- [Multimodal Fusion Interactions: A Study of Human and Automatic Quantification](https://mlsanity.com/paper/2306.04125/) (2023)
- [3MASSIV: Multilingual, Multimodal and Multi-Aspect dataset of Social Media Short Videos](https://mlsanity.com/paper/2203.14456/) (2022)
- [Verifying Cross-modal Entity Consistency in News using Vision-language Models](https://mlsanity.com/paper/2501.11403/) (2025)
- [Overview of ImageArg-2023: The First Shared Task in Multimodal Argument Mining](https://mlsanity.com/paper/2310.12172/) (2023)
- [Looking Beyond the Obvious: A Survey on Abstract Concept Recognition for Video Understanding](https://mlsanity.com/paper/2508.20765/) (2025)
- [On the Complementarity of Images and Text for the Expression of Emotions in Social Media](https://mlsanity.com/paper/2202.07427/) (2022)
- [DM2S2: Deep Multi-Modal Sequence Sets with Hierarchical Modality Attention](https://mlsanity.com/paper/2209.03126/) (2022)
- [Enhancing Cross-Modal Contextual Congruence for Crowdfunding Success using Knowledge-infused Learning](https://mlsanity.com/paper/2402.03607/) (2024)
- [AnANet: Modeling Association and Alignment for Cross-modal Correlation Classification](https://mlsanity.com/paper/2109.00693/) (2021)
- [Do Vision-Language Models Understand Visual Persuasiveness? A Diagnosis via Visual Persuasive Factors](https://mlsanity.com/paper/2511.17036/) (2025)

## Abstract (arXiv)

Images and text in advertisements interact in complex, non-literal ways. The two channels are usually complementary, with each channel telling a different part of the story. Current approaches, such as image captioning methods, only examine literal, redundant relationships, where image and text show exactly the same content. To understand more complex relationships, we first collect a dataset of advertisement interpretations for whether the image and slogan in the same visual advertisement form a parallel (conveying the same message without literally saying the same thing) or non-parallel relationship, with the help of workers recruited on Amazon Mechanical Turk. We develop a variety of features that capture the creativity of images and the specificity or ambiguity of text, as well as methods that analyze the semantics within and across channels. We show that our method outperforms standard image-text alignment approaches on predicting the parallel/non-parallel relationship between image and text.