# A Quantum Probability Driven Framework for Joint Multi-Modal Sarcasm, Sentiment and Emotion Analysis

arXiv: https://arxiv.org/abs/2306.03650 · HTML: https://mlsanity.com/paper/2306.03650/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 29 references that are in the map.
- First posted on arXiv 6 Jun 2023.

## Builds on (most cited first, 8 of 8)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations](https://mlsanity.com/paper/1810.02508/) (2018)
- [MMGCN: Multimodal Fusion via Deep Graph Convolution Network for Emotion Recognition in Conversation](https://mlsanity.com/paper/2107.06779/) (2021)
- [A Transformer-based approach to Irony and Sarcasm detection](https://mlsanity.com/paper/1911.10401/) (2019)
- [CNM: An Interpretable Complex-valued Network for Matching](https://mlsanity.com/paper/1904.05298/) (2019)
- [UPB at SemEval-2020 Task 8: Joint Textual and Visual Modeling in a Multi-Task Learning Architecture for Memotion Analysis](https://mlsanity.com/paper/2009.02779/) (2020)
- [Multi-Modal Sarcasm Detection Based on Contrastive Attention Mechanism](https://mlsanity.com/paper/2109.15153/) (2021)

## Built on by (most cited first, 9 of 9)

- [DialogueLLM: Context and Emotion Knowledge-Tuned Large Language Models for Emotion Recognition in Conversations](https://mlsanity.com/paper/2310.11374/) (2023)
- [InterCLIP-MEP: Interactive CLIP and Memory-Enhanced Predictor for Multi-modal Sarcasm Detection](https://mlsanity.com/paper/2406.16464/) (2024)
- [FanChuan: A Multilingual and Graph-Structured Benchmark For Parody Detection and Analysis](https://mlsanity.com/paper/2502.16503/) (2025)
- [Spoken in Jest, Detected in Earnest: A Systematic Review of Sarcasm Recognition -- Multimodal Fusion, Challenges, and Future Prospects](https://mlsanity.com/paper/2509.04605/) (2025)
- [Commander-GPT: Dividing and Routing for Multimodal Sarcasm Detection](https://mlsanity.com/paper/2506.19420/) (2025)
- [Emotion Recognition in Older Adults with Quantum Machine Learning and Wearable Sensors](https://mlsanity.com/paper/2507.08175/) (2025)
- [Conditional Information Bottleneck for Multimodal Fusion: Overcoming Shortcut Learning in Sarcasm Detection](https://mlsanity.com/paper/2508.10644/) (2025)
- [QiNN-QJ: A Quantum-inspired Neural Network with Quantum Jump for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2510.27091/) (2025)
- [A GAN and LLM-Driven Data Augmentation Framework for Dynamic Linguistic Pattern Modeling in Chinese Sarcasm Detection](https://mlsanity.com/paper/2604.08381/) (2026)

## Abstract (arXiv)

Sarcasm, sentiment, and emotion are three typical kinds of spontaneous affective responses of humans to external events and they are tightly intertwined with each other. Such events may be expressed in multiple modalities (e.g., linguistic, visual and acoustic), e.g., multi-modal conversations. Joint analysis of humans' multi-modal sarcasm, sentiment, and emotion is an important yet challenging topic, as it is a complex cognitive process involving both cross-modality interaction and cross-affection correlation. From the probability theory perspective, cross-affection correlation also means that the judgments on sarcasm, sentiment, and emotion are incompatible. However, this exposed phenomenon cannot be sufficiently modelled by classical probability theory due to its assumption of compatibility. Neither do the existing approaches take it into consideration. In view of the recent success of quantum probability (QP) in modeling human cognition, particularly contextual incompatible decision making, we take the first step towards introducing QP into joint multi-modal sarcasm, sentiment, and emotion analysis. Specifically, we propose a QUantum probabIlity driven multi-modal sarcasm, sEntiment and emoTion analysis framework, termed QUIET. Extensive experiments on two datasets and the results show that the effectiveness and advantages of QUIET in comparison with a wide range of the state-of-the-art baselines. We also show the great potential of QP in multi-affect analysis.