# Emotion-Aware Multimodal Fusion for Meme Emotion Detection

arXiv: https://arxiv.org/abs/2403.10279 · HTML: https://mlsanity.com/paper/2403.10279/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Mar 2024.

## Built on by (most cited first, 7 of 7)

- [A Multimodal-Multitask Framework with Cross-modal Relation and Hierarchical Interactive Attention for Semantic Comprehension](https://mlsanity.com/paper/2508.16300/) (2025)
- [Meme Similarity and Emotion Detection using Multimodal Analysis](https://mlsanity.com/paper/2503.17493/) (2025)
- [MER-Bench: A Comprehensive Benchmark for Multimodal Meme Reappraisal](https://mlsanity.com/paper/2603.15020/) (2026)
- [Decoding Memes: Benchmarking Narrative Role Classification across Multilingual and Multimodal Models](https://mlsanity.com/paper/2506.23122/) (2025)
- [Edu-EmotionNet: Cross-Modality Attention Alignment with Temporal Feedback Loops](https://mlsanity.com/paper/2510.08802/) (2025)
- [Distinguishing Right from Wrong in Debates: Attribution Analysis of Chinese Harmful Memes](https://mlsanity.com/paper/2605.24344/) (2026)
- [Emotion Experience, Expression, and Perception: Emotion Analysis on Multimodal Social Media Posts](https://mlsanity.com/paper/2609.18385/) (2026)

## Abstract (arXiv)

The ever-evolving social media discourse has witnessed an overwhelming use of memes to express opinions or dissent. Besides being misused for spreading malcontent, they are mined by corporations and political parties to glean the public's opinion. Therefore, memes predominantly offer affect-enriched insights towards ascertaining the societal psyche. However, the current approaches are yet to model the affective dimensions expressed in memes effectively. They rely extensively on large multimodal datasets for pre-training and do not generalize well due to constrained visual-linguistic grounding. In this paper, we introduce MOOD (Meme emOtiOns Dataset), which embodies six basic emotions. We then present ALFRED (emotion-Aware muLtimodal Fusion foR Emotion Detection), a novel multimodal neural framework that (i) explicitly models emotion-enriched visual cues, and (ii) employs an efficient cross-modal fusion via a gating mechanism. Our investigation establishes ALFRED's superiority over existing baselines by 4.94% F1. Additionally, ALFRED competes strongly with previous best approaches on the challenging Memotion task. We then discuss ALFRED's domain-agnostic generalizability by demonstrating its dominance on two recently-released datasets - HarMeme and Dank Memes, over other baselines. Further, we analyze ALFRED's interpretability using attention maps. Finally, we highlight the inherent challenges posed by the complex interplay of disparate modality-specific cues toward meme analysis.