# M-QUEST -- Meme Question-Understanding Evaluation on Semantics and Toxicity

arXiv: https://arxiv.org/abs/2603.03315 · HTML: https://mlsanity.com/paper/2603.03315/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 58 references that are in the map.
- First posted on arXiv 9 Feb 2026.

## Builds on (most cited first, 18 of 18)

- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes](https://mlsanity.com/paper/2005.04790/) (2020)
- [Social Chemistry 101: Learning to Reason about Social and Moral Norms](https://mlsanity.com/paper/2011.00620/) (2020)
- [SemEval-2020 Task 8: Memotion Analysis -- The Visuo-Lingual Metaphor!](https://mlsanity.com/paper/2008.03781/) (2020)
- [GOAT-Bench: Safety Insights to Large Multimodal Models through Meme-Based Social Abuse](https://mlsanity.com/paper/2401.01523/) (2024)
- [Detecting Propaganda Techniques in Memes](https://mlsanity.com/paper/2109.08013/) (2021)
- [Decoding the Underlying Meaning of Multimodal Hateful Memes](https://mlsanity.com/paper/2305.17678/) (2023)
- [Hate Speech in Pixels: Detection of Offensive Memes towards Automatic Moderation](https://mlsanity.com/paper/1910.02334/) (2019)
- [On the Evolution of (Hateful) Memes by Means of Multimodal Contrastive Learning](https://mlsanity.com/paper/2212.06573/) (2022)
- [DISARM: Detecting the Victims Targeted by Harmful Memes](https://mlsanity.com/paper/2205.05738/) (2022)
- [BanglaAbuseMeme: A Dataset for Bengali Abusive Meme Classification](https://mlsanity.com/paper/2310.11748/) (2023)
- [DisinfoMeme: A Multimodal Dataset for Detecting Meme Intentionally Spreading Out Disinformation](https://mlsanity.com/paper/2205.12617/) (2022)
- [Meme-ingful Analysis: Enhanced Understanding of Cyberbullying in Memes Through Multimodal Explanations](https://mlsanity.com/paper/2401.09899/) (2024)
- [Benchmark dataset of memes with text transcriptions for automatic detection of multi-modal misogynistic content](https://mlsanity.com/paper/2106.08409/) (2021)
- [A Template Is All You Meme](https://mlsanity.com/paper/2311.06649/) (2023)
- [Hate Me Not: Detecting Hate Inducing Memes in Code Switched Languages](https://mlsanity.com/paper/2204.11356/) (2022)
- [A Study of Commonsense Reasoning over Visual Object Properties](https://mlsanity.com/paper/2508.10956/) (2025)

## Built on by (most cited first, 3 of 3)

- [MemeBench: What LVLMs Miss When Interpreting Culture-Dependent Memes](https://mlsanity.com/paper/2607.27798/) (2026)
- [Beyond the Literal: Decomposing Pragmatic Intent in Multimodal Meme Understanding](https://mlsanity.com/paper/2606.03604/) (2026)
- [Learn Before You Judge: Progressive Knowledge-to-Decision Alignment for Explainable Hateful Meme Detection](https://mlsanity.com/paper/2609.19778/) (2026)

## Abstract (arXiv)

Internet memes are a powerful form of online communication, yet their nature and reliance on commonsense knowledge make toxicity detection challenging. Identifying key features for meme interpretation and understanding, is a crucial task. Previous work has been focused on some elements contributing to the meaning, such as the Textual dimension via OCR, the Visual dimension via object recognition, upper layers of meaning like the Emotional dimension, Toxicity detection via proxy variables, such as hate speech detection, and sentiment analysis. Nevertheless, there is still a lack of an overall architecture able to formally identify elements contributing to the meaning of a meme, and be used in the sense-making process. In this work, we present a semantic framework and a corresponding benchmark for automatic knowledge extraction from memes. First, we identify the necessary dimensions to understand and interpret a meme: Textual material, Visual material, Scene, Background Knowledge, Emotion, Semiotic Projection, Analogical Mapping, Overall Intent, Target Community, and Toxicity Assessment. Second, the framework guides a semi-automatic process of generating a benchmark with commonsense question-answer pairs about meme toxicity assessment and its underlying reason. The resulting benchmark M-QUEST consists of 609 question-answer pairs for 307 memes. Thirdly, we evaluate eight open-source large language models on their ability to correctly solve M-QUEST. Our results show that current models' commonsense reasoning capabilities for toxic meme interpretation vary depending on the dimension and architecture. Models with instruction tuning and reasoning capabilities significantly outperform the others, though pragmatic inference questions remain challenging. We release code, benchmark, and prompts to support future research intersecting multimodal content safety and commonsense reasoning.