# Deep Learning Approaches for Multimodal Intent Recognition: A Survey

arXiv: https://arxiv.org/abs/2507.22934 · HTML: https://mlsanity.com/paper/2507.22934/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 52 of its 165 references that are in the map.
- First posted on arXiv 24 Jul 2025.

## Builds on (most cited first, 30 of 52)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling](https://mlsanity.com/paper/1810.00278/) (2018)
- [Efficient Intent Detection with Dual Sentence Encoders](https://mlsanity.com/paper/2003.04807/) (2020)
- [An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction](https://mlsanity.com/paper/1909.02027/) (2019)
- [Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces](https://mlsanity.com/paper/1805.10190/) (2018)
- [MultiWOZ 2.2 : A Dialogue Dataset with Additional Annotation Corrections and State Tracking Baselines](https://mlsanity.com/paper/2007.12720/) (2020)
- [SLURP: A Spoken Language Understanding Resource Package](https://mlsanity.com/paper/2011.13205/) (2020)
- [BERT for Joint Intent Classification and Slot Filling](https://mlsanity.com/paper/1902.10909/) (2019)
- [Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Filling](https://mlsanity.com/paper/1609.01454/) (2016)
- [Intent Contrastive Learning for Sequential Recommendation](https://mlsanity.com/paper/2202.02519/) (2022)
- [A Fine-tuned Wav2vec 2.0/HuBERT Benchmark For Speech Emotion Recognition, Speaker Verification and Spoken Language Understanding](https://mlsanity.com/paper/2111.02735/) (2021)
- [Out-of-domain Detection for Natural Language Understanding in Dialog Systems](https://mlsanity.com/paper/1909.03862/) (2019)
- [Out-of-Scope Intent Detection with Self-Supervision and Discriminative Training](https://mlsanity.com/paper/2106.08616/) (2021)
- [MIntRec: A New Dataset for Multimodal Intent Recognition](https://mlsanity.com/paper/2209.04355/) (2022)
- [Integrating Text and Image: Determining Multimodal Document Intent in Instagram Posts](https://mlsanity.com/paper/1904.09073/) (2019)
- [Behavioral Intention Prediction in Driving Scenes: A Survey](https://mlsanity.com/paper/2211.00385/) (2022)
- [MIntRec2.0: A Large-scale Benchmark Dataset for Multimodal Intent Recognition and Out-of-scope Detection in Conversations](https://mlsanity.com/paper/2403.10943/) (2024)
- [Is Conversational XAI All You Need? Human-AI Decision Making With a Conversational XAI Assistant](https://mlsanity.com/paper/2501.17546/) (2025)
- [Cascade and Parallel Convolutional Recurrent Neural Networks on EEG-based Intention Recognition for Brain Computer Interface](https://mlsanity.com/paper/1708.06578/) (2017)
- [TEXTOIR: An Integrated and Visualized Platform for Text Open Intent Recognition](https://mlsanity.com/paper/2110.15063/) (2021)
- [Intentonomy: a Dataset and Study towards Human Intent Understanding](https://mlsanity.com/paper/2011.05558/) (2020)
- [Emotion and Intent Joint Understanding in Multimodal Conversation: A Benchmarking Dataset](https://mlsanity.com/paper/2407.02751/) (2024)
- [Knowledge Distillation from BERT Transformer to Speech Transformer for Intent Classification](https://mlsanity.com/paper/2108.02598/) (2021)

## Built on by (most cited first, 6 of 6)

- [MVCL-DAF++: Enhancing Multimodal Intent Recognition via Prototype-Aware Contrastive Alignment and Coarse-to-Fine Dynamic Attention Fusion](https://mlsanity.com/paper/2509.17446/) (2025)
- [IntentVLM: Open-Vocabulary Intention Recognition through Forward-Inverse Modeling with Video-Language Models](https://mlsanity.com/paper/2604.24002/) (2026)
- [Moltbook Moderation: Uncovering Hidden Intent Through Multi-Turn Dialogue](https://mlsanity.com/paper/2605.12856/) (2026)
- [Adaptive Modality Reliability Diagnosis and Restoration for Robust Multimodal Intent Recognition](https://mlsanity.com/paper/2608.03475/) (2026)
- [Modality Agreement- and Conflict-Aware Prototype Hypergraph Learning for Multimodal Intent Understanding](https://mlsanity.com/paper/2608.04054/) (2026)
- [Neuro-Symbolic Graph Transformers for Hierarchical Intention Recognition from Multimodal Human Activity Sequences](https://mlsanity.com/paper/2610.18176/) (2026)

## Abstract (arXiv)

Intent recognition aims to identify users' underlying intentions, traditionally focusing on text in natural language processing. With growing demands for natural human-computer interaction, the field has evolved through deep learning and multimodal approaches, incorporating data from audio, vision, and physiological signals. Recently, the introduction of Transformer-based models has led to notable breakthroughs in this domain. This article surveys deep learning methods for intent recognition, covering the shift from unimodal to multimodal techniques, relevant datasets, methodologies, applications, and current challenges. It provides researchers with insights into the latest developments in multimodal intent recognition (MIR) and directions for future research.