mlsanityData to 7 Oct 2026

Towards Multi-Modal Sarcasm Detection via Hierarchical Congruity Modeling with Knowledge Enhancement

arXiv 2210.03501 ↗First posted 7 Oct 2022cs.CLcited by 26

Not on the map. Why?

Abstract (arXiv)

Sarcasm is a linguistic phenomenon indicating a discrepancy between literal meanings and implied intentions. Due to its sophisticated nature, it is usually challenging to be detected from the text itself. As a result, multi-modal sarcasm detection has received more attention in both academia and industries. However, most existing techniques only modeled the atomic-level inconsistencies between the text input and its accompanying image, ignoring more complex compositions for both modalities. Moreover, they neglected the rich information contained in external knowledge, e.g., image captions. In this paper, we propose a novel hierarchical framework for sarcasm detection by exploring both the atomic-level congruity based on multi-head cross attention mechanism and the composition-level congruity based on graph neural networks, where a post with low congruity can be identified as sarcasm. In addition, we exploit the effect of various knowledge resources for sarcasm detection. Evaluation results on a public multi-modal sarcasm detection dataset based on Twitter demonstrate the superiority of our proposed model.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 12 of its 41 references that are in the map. First posted on arXiv 7 Oct 2022.

Builds on

12 of 41 references.

References · 12

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2017Graph Attention Networkscited by 8,854
  7. 2014Convolutional Neural Networks for Sentence Classificationcited by 1,124
  8. 2021ClipCap: CLIP Prefix for Image Captioningcited by 557
  9. 2017Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasmcited by 117
  10. 2020Graph Structured Network for Image-Text Matchingcited by 93
  11. 2016Detecting Sarcasm in Multimodal Social Platformscited by 43
  12. 2018Reasoning with Sarcasm by Reading In-betweencited by 38

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 7; 2024: 9; 2025: 7; 2026: 3.

26 papers.

Sort:

By area:

  1. 2023MMSD2.0: Towards a Reliable Multi-modal Sarcasm Detection Systemcited by 31
  2. 2023MMoE: Enhancing Multimodal Models with Mixtures of Multimodal Interaction Expertscited by 21
  3. 2024A Survey of Multimodal Sarcasm Detectioncited by 19
  4. 2023Interpretable Multimodal Misinformation Detection with Logic Reasoningcited by 16
  5. 2024Can Large Multimodal Models Uncover Deep Semantics Behind Images?cited by 16
  6. 2023Debiasing Multimodal Sarcasm Detection with Contrastive Learningcited by 12
  7. 2024CofiPara: A Coarse-to-fine Paradigm for Multimodal Sarcasm Target Identification with Large Multimodal Modelscited by 8
  8. 2023Alternative Telescopic Displacement: An Efficient Multimodal Alignment Methodcited by 6
  9. 2024Mixture-of-Prompt-Experts for Multi-modal Semantic Understandingcited by 6
  10. 2025Exploring Cognitive and Aesthetic Causality for Multimodal Aspect-Based Sentiment Analysiscited by 6
  11. 2025Beyond Spurious Signals: Debiasing Multimodal Large Language Models via Counterfactual Inference and Adaptive Expert Routingcited by 5
  12. 2023Evaluating Picture Description Speech for Dementia Detection using Image-text Alignmentcited by 4
14 more
  1. 2024InterCLIP-MEP: Interactive CLIP and Memory-Enhanced Predictor for Multi-modal Sarcasm Detectioncited by 4
  2. 2024RCLMuFN: Relational Context Learning and Multiplex Fusion Network for Multimodal Sarcasm Detectioncited by 4
  3. 2025Seeing Sarcasm Through Different Eyes: Analyzing Multimodal Sarcasm Perception in Large Vision-Language Modelscited by 4
  4. 2024MIKO: Multimodal Intention Knowledge Distillation from Large Language Models for Social-Media Commonsense Discoverycited by 3
  5. 2024Multi-modal Semantic Understanding with Contrastive Cross-modal Feature Alignmentcited by 3
  6. 2025MODA: MOdular Duplex Attention for Multimodal Perception, Cognition, and Emotion Understandingcited by 2
  7. 2023Image Matters: A New Dataset and Empirical Study for Multimodal Hyperbole Detectioncited by 1
  8. 2025Dual Modality-Aware Gated Prompt Tuning for Few-Shot Multimodal Sarcasm Detectioncited by 1
  9. 2025Adapter-state Sharing CLIP for Parameter-efficient Multimodal Sarcasm Detectioncited by 1
  10. 2026GDCNet: Generative Discrepancy Comparison Network for Multimodal Sarcasm Detectioncited by 1
  11. 2024An Innovative CGL-MHA Model for Sarcasm Sentiment Recognition Using the MindSpore Frameworkcited by 0
  12. 2025Teaching Sarcasm: Few-Shot Multimodal Sarcasm Detection via Distillation to a Parameter-Efficient Studentcited by 0
  13. 2026Can a Unimodal Language Agent Provide Preferences to Tune a Multimodal Vision-Language Model?cited by 0
  14. 2026Computational Humor with Multimodal LLMs: Methods, Datasets, Evaluation, and Challengescited by 0