# Emotion and Intention Guided Multi-Modal Learning for Sticker Response Selection

arXiv: https://arxiv.org/abs/2511.17587 · HTML: https://mlsanity.com/paper/2511.17587/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 46 references that are in the map.
- First posted on arXiv 16 Nov 2025.

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Align before Fuse: Vision and Language Representation Learning with Momentum Distillation](https://mlsanity.com/paper/2107.07651/) (2021)
- [Affective Image Content Analysis: Two Decades Review and New Perspectives](https://mlsanity.com/paper/2106.16125/) (2021)
- [NoteLLM-2: Multimodal Large Representation Models for Recommendation](https://mlsanity.com/paper/2405.16789/) (2024)
- [G3: An Effective and Adaptive Framework for Worldwide Geolocalization Using Large Multi-Modality Models](https://mlsanity.com/paper/2405.14702/) (2024)
- [IMF: Interactive Multimodal Fusion Model for Link Prediction](https://mlsanity.com/paper/2303.10816/) (2023)
- [Image-Question-Answer Synergistic Network for Visual Dialog](https://mlsanity.com/paper/1902.09774/) (2019)
- [STICKERCONV: Generating Multimodal Empathetic Responses from Scratch](https://mlsanity.com/paper/2402.01679/) (2024)
- [Learning to Respond with Stickers: A Framework of Unifying Multi-Modality in Multi-Turn Dialog](https://mlsanity.com/paper/2003.04679/) (2020)
- [Towards Expressive Communication with Internet Memes: A New Multimodal Conversation Dataset and Benchmark](https://mlsanity.com/paper/2109.01839/) (2021)
- [Improving Empathetic Dialogue Generation by Dynamically Infusing Commonsense Knowledge](https://mlsanity.com/paper/2306.04657/) (2023)
- [APTNESS: Incorporating Appraisal Theory and Emotion Support Strategies for Empathetic Response Generation](https://mlsanity.com/paper/2407.21048/) (2024)
- [Large Multimodal Model Compression via Efficient Pruning and Distillation at AntGroup](https://mlsanity.com/paper/2312.05795/) (2023)
- [Understanding Chat Messages for Sticker Recommendation in Messaging Apps](https://mlsanity.com/paper/1902.02704/) (2019)
- [FindRec: Stein-Guided Entropic Flow for Multi-Modal Sequential Recommendation](https://mlsanity.com/paper/2507.04651/) (2025)
- [Learning to Respond with Your Favorite Stickers: A Framework of Unifying Multi-Modality and User Preference in Multi-Turn Dialog](https://mlsanity.com/paper/2011.03322/) (2020)
- [Reply with Sticker: New Dataset and Model for Sticker Retrieval](https://mlsanity.com/paper/2403.05427/) (2024)
- [MGHFT: Multi-Granularity Hierarchical Fusion Transformer for Cross-Modal Sticker Emotion Recognition](https://mlsanity.com/paper/2507.18929/) (2025)
- [Towards Real-World Stickers Use: A New Dataset for Multi-Tag Sticker Recognition](https://mlsanity.com/paper/2403.05428/) (2024)
- [Empowering Large Language Model for Sequential Recommendation via Multimodal Embeddings and Semantic IDs](https://mlsanity.com/paper/2509.02017/) (2025)

## Built on by (most cited first, 2 of 2)

- [Empowering Cross-Domain Sequential Recommendation with Hybrid Tokenization and Serial-Parallel Decoding](https://mlsanity.com/paper/2607.28659/) (2026)
- [ESCRAG-R1: Retrieval-Augmented Reinforcement Learning for Emotional Support Conversation](https://mlsanity.com/paper/2608.21925/) (2026)

## Abstract (arXiv)

Stickers are widely used in online communication to convey emotions and implicit intentions. The Sticker Response Selection (SRS) task aims to select the most contextually appropriate sticker based on the dialogue. However, existing methods typically rely on semantic matching and model emotional and intentional cues separately, which can lead to mismatches when emotions and intentions are misaligned. To address this issue, we propose Emotion and Intention Guided Multi-Modal Learning (EIGML). This framework is the first to jointly model emotion and intention, effectively reducing the bias caused by isolated modeling and significantly improving selection accuracy. Specifically, we introduce Dual-Level Contrastive Framework to perform both intra-modality and inter-modality alignment, ensuring consistent representation of emotional and intentional features within and across modalities. In addition, we design an Intention-Emotion Guided Multi-Modal Fusion module that integrates emotional and intentional information progressively through three components: Emotion-Guided Intention Knowledge Selection, Intention-Emotion Guided Attention Fusion, and Similarity-Adjusted Matching Mechanism. This design injects rich, effective information into the model and enables a deeper understanding of the dialogue, ultimately enhancing sticker selection performance. Experimental results on two public SRS datasets show that EIGML consistently outperforms state-of-the-art baselines, achieving higher accuracy and a better understanding of emotional and intentional features. Code is provided in the supplementary materials.