# Figurative-cum-Commonsense Knowledge Infusion for Multimodal Mental Health Meme Classification

arXiv: https://arxiv.org/abs/2501.15321 · HTML: https://mlsanity.com/paper/2501.15321/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 48 references that are in the map.
- First posted on arXiv 25 Jan 2025.

## Builds on (most cited first, 13 of 13)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [COMET: Commonsense Transformers for Automatic Knowledge Graph Construction](https://mlsanity.com/paper/1906.05317/) (2019)
- [MentalBERT: Publicly Available Pretrained Language Models for Mental Healthcare](https://mlsanity.com/paper/2110.15621/) (2021)
- [MOMENTA: A Multimodal Framework for Detecting Harmful Memes and Their Targets](https://mlsanity.com/paper/2109.05184/) (2021)
- [Detecting and Understanding Harmful Memes: A Survey](https://mlsanity.com/paper/2205.04274/) (2022)
- [Speaker and Time-aware Joint Contextual Learning for Dialogue-act Classification in Counselling Conversations](https://mlsanity.com/paper/2111.06647/) (2021)
- [Knowledge Planning in Large Language Models for Domain-Aligned Counseling Summarization](https://mlsanity.com/paper/2409.14907/) (2024)
- [WorryWords: Norms of Anxiety Association for over 44k English Words](https://mlsanity.com/paper/2411.03966/) (2024)
- [A Computational Approach to Understand Mental Health from Reddit: Knowledge-aware Multitask Learning Framework](https://mlsanity.com/paper/2203.11856/) (2022)
- [Critical Behavioral Traits Foster Peer Engagement in Online Mental Health Communities](https://mlsanity.com/paper/2309.01618/) (2023)
- [Towards a Generative Approach for Emotion Detection and Reasoning](https://mlsanity.com/paper/2408.04906/) (2024)

## Built on by (most cited first, 6 of 6)

- [M-HELP: Using Social Media Data to Detect Mental Health Help-Seeking Signals](https://mlsanity.com/paper/2508.15440/) (2025)
- [MemeTAG: Keyword-Driven Meme Classification through Tag Embedding Reconstruction](https://mlsanity.com/paper/2609.20962/) (2026)
- [All Changes May Have Invariant Principles: Improving Ever-Shifting Harmful Meme Detection via Design Concept Reproduction](https://mlsanity.com/paper/2601.04567/) (2026)
- [Fall into a Pit, Gain in a Wit: Cognitive-Guided Harmful Meme Detection via Misjudgment Risk Pattern Retrieval](https://mlsanity.com/paper/2510.15946/) (2025)
- [FigSIM: A Dataset for Fine-grained Suicide Severity and Figurative Language in Suicide Memes](https://mlsanity.com/paper/2606.02523/) (2026)
- [Towards High-Level Semantic Intelligence](https://mlsanity.com/paper/2607.24082/) (2026)

## Abstract (arXiv)

The expression of mental health symptoms through non-traditional means, such as memes, has gained remarkable attention over the past few years, with users often highlighting their mental health struggles through figurative intricacies within memes. While humans rely on commonsense knowledge to interpret these complex expressions, current Multimodal Language Models (MLMs) struggle to capture these figurative aspects inherent in memes. To address this gap, we introduce a novel dataset, AxiOM, derived from the GAD anxiety questionnaire, which categorizes memes into six fine-grained anxiety symptoms. Next, we propose a commonsense and domain-enriched framework, M3H, to enhance MLMs' ability to interpret figurative language and commonsense knowledge. The overarching goal remains to first understand and then classify the mental health symptoms expressed in memes. We benchmark M3H against 6 competitive baselines (with 20 variations), demonstrating improvements in both quantitative and qualitative metrics, including a detailed human evaluation. We observe a clear improvement of 4.20% and 4.66% on weighted-F1 metric. To assess the generalizability, we perform extensive experiments on a public dataset, RESTORE, for depressive symptom identification, presenting an extensive ablation study that highlights the contribution of each module in both datasets. Our findings reveal limitations in existing models and the advantage of employing commonsense to enhance figurative understanding.