# All Changes May Have Invariant Principles: Improving Ever-Shifting Harmful Meme Detection via Design Concept Reproduction

arXiv: https://arxiv.org/abs/2601.04567 · HTML: https://mlsanity.com/paper/2601.04567/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 44 references that are in the map.
- First posted on arXiv 8 Jan 2026.

## Builds on (most cited first, 25 of 25)

- [ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks](https://mlsanity.com/paper/1908.02265/) (2019)
- [VisualBERT: A Simple and Performant Baseline for Vision and Language](https://mlsanity.com/paper/1908.03557/) (2019)
- [The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes](https://mlsanity.com/paper/2005.04790/) (2020)
- [Graph Retrieval-Augmented Generation: A Survey](https://mlsanity.com/paper/2408.08921/) (2024)
- [SEED-Bench-2-Plus: Benchmarking Multimodal Large Language Models with Text-Rich Visual Comprehension](https://mlsanity.com/paper/2404.16790/) (2024)
- [Supervised Multimodal Bitransformers for Classifying Images and Text](https://mlsanity.com/paper/1909.02950/) (2019)
- [MOMENTA: A Multimodal Framework for Detecting Harmful Memes and Their Targets](https://mlsanity.com/paper/2109.05184/) (2021)
- [Prompting for Multimodal Hateful Meme Classification](https://mlsanity.com/paper/2302.04156/) (2023)
- [Detecting and Understanding Harmful Memes: A Survey](https://mlsanity.com/paper/2205.04274/) (2022)
- [GOAT-Bench: Safety Insights to Large Multimodal Models through Meme-Based Social Abuse](https://mlsanity.com/paper/2401.01523/) (2024)
- [Vilio: State-of-the-art Visio-Linguistic Models applied to Hateful Memes](https://mlsanity.com/paper/2012.07788/) (2020)
- [A Multimodal Framework for the Detection of Hateful Memes](https://mlsanity.com/paper/2012.12871/) (2020)
- [Pro-Cap: Leveraging a Frozen Vision-Language Model for Hateful Meme Detection](https://mlsanity.com/paper/2308.08088/) (2023)
- [Disentangling Hate in Online Memes](https://mlsanity.com/paper/2108.06207/) (2021)
- [On Explaining Multimodal Hateful Meme Detection Models](https://mlsanity.com/paper/2204.01734/) (2022)
- [Towards Low-Resource Harmful Meme Detection with LMM Agents](https://mlsanity.com/paper/2411.05383/) (2024)
- [From GPT-4 to Gemini and Beyond: Assessing the Landscape of MLLMs on Generalizability, Trustworthiness and Causality through Four Modalities](https://mlsanity.com/paper/2401.15071/) (2024)
- [MIND: A Multi-agent Framework for Zero-shot Harmful Meme Detection](https://mlsanity.com/paper/2507.06908/) (2025)
- [Toxic Memes: A Survey of Computational Perspectives on the Detection and Explanation of Meme Toxicities](https://mlsanity.com/paper/2406.07353/) (2024)
- [Robust Adaptation of Large Multimodal Models for Retrieval Augmented Hateful Meme Detection](https://mlsanity.com/paper/2502.13061/) (2025)
- [Systematic Literature Reviews in Software Engineering -- Enhancement of the Study Selection Process using Cohen's Kappa Statistic](https://mlsanity.com/paper/2005.10890/) (2020)
- [AdamMeme: Adaptively Probe the Reasoning Capacity of Multimodal Large Language Models on Harmfulness](https://mlsanity.com/paper/2507.01702/) (2025)
- [Figurative-cum-Commonsense Knowledge Infusion for Multimodal Mental Health Meme Classification](https://mlsanity.com/paper/2501.15321/) (2025)
- [Entropy and complexity unveil the landscape of memes evolution](https://mlsanity.com/paper/2105.12376/) (2021)
- [Bootstrapping Heterogeneous Graph Representation Learning via Large Language Models: A Generalized Approach](https://mlsanity.com/paper/2412.08038/) (2024)

## Built on by (most cited first, 1 of 1)

- [SheetCompass: Hierarchical Relation Graphs for Agentic Spreadsheet Reasoning](https://mlsanity.com/paper/2608.14452/) (2026)

## Abstract (arXiv)

Harmful memes are ever-shifting in the Internet communities, which are difficult to analyze due to their type-shifting and temporal-evolving nature. Although these memes are shifting, we find that different memes may share invariant principles, i.e., the underlying design concept of malicious users, which can help us analyze why these memes are harmful. In this paper, we propose RepMD, an ever-shifting harmful meme detection method based on the design concept reproduction. We first refer to the attack tree to define the Design Concept Graph (DCG), which describes steps that people may take to design a harmful meme. Then, we derive the DCG from historical memes with design step reproduction and graph pruning. Finally, we use DCG to guide the Multimodal Large Language Model (MLLM) to detect harmful memes. The evaluation results show that RepMD achieves the highest accuracy with 81.1% and has slight accuracy decreases when generalized to type-shifting and temporal-evolving memes. Human evaluation shows that RepMD can improve the efficiency of human discovery on harmful memes, with 15~30 seconds per meme.