mlsanityData to 14 Oct 2026

Deep Learning Approaches for Multimodal Intent Recognition: A Survey

arXiv 2507.22934 ↗First posted 24 Jul 2025cs.CL cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

Intent recognition aims to identify users' underlying intentions, traditionally focusing on text in natural language processing. With growing demands for natural human-computer interaction, the field has evolved through deep learning and multimodal approaches, incorporating data from audio, vision, and physiological signals. Recently, the introduction of Transformer-based models has led to notable breakthroughs in this domain. This article surveys deep learning methods for intent recognition, covering the shift from unimodal to multimodal techniques, relevant datasets, methodologies, applications, and current challenges. It provides researchers with insights into the latest developments in multimodal intent recognition (MIR) and directions for future research.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 52 of its 165 references that are in the map. First posted on arXiv 24 Jul 2025.

Builds on

52 of 165 references.

References · 52

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  5. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  6. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  7. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  8. 2014Convolutional Neural Networks for Sentence Classificationcited by 1,127
  9. 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 766
  10. 2020Efficient Intent Detection with Dual Sentence Encoderscited by 433
  11. 2019An Evaluation Dataset for Intent Classification and Out-of-Scope Predictioncited by 386
  12. 2018Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfacescited by 377
38 more
  1. 2020MultiWOZ 2.2 : A Dialogue Dataset with Additional Annotation Corrections and State Tracking Baselinescited by 204
  2. 2020SLURP: A Spoken Language Understanding Resource Packagecited by 198
  3. 2019BERT for Joint Intent Classification and Slot Fillingcited by 196
  4. 2016Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Fillingcited by 137
  5. 2022Intent Contrastive Learning for Sequential Recommendationcited by 91
  6. 2021A Fine-tuned Wav2vec 2.0/HuBERT Benchmark For Speech Emotion Recognition, Speaker Verification and Spoken Language Understandingcited by 82
  7. 2019Out-of-domain Detection for Natural Language Understanding in Dialog Systemscited by 62
  8. 2021Out-of-Scope Intent Detection with Self-Supervision and Discriminative Trainingcited by 42
  9. 2022MIntRec: A New Dataset for Multimodal Intent Recognitioncited by 41
  10. 2019Integrating Text and Image: Determining Multimodal Document Intent in Instagram Postscited by 37
  11. 2022Behavioral Intention Prediction in Driving Scenes: A Surveycited by 30
  12. 2024MIntRec2.0: A Large-scale Benchmark Dataset for Multimodal Intent Recognition and Out-of-scope Detection in Conversationscited by 26
  13. 2025Is Conversational XAI All You Need? Human-AI Decision Making With a Conversational XAI Assistantcited by 26
  14. 2017Cascade and Parallel Convolutional Recurrent Neural Networks on EEG-based Intention Recognition for Brain Computer Interfacecited by 24
  15. 2021TEXTOIR: An Integrated and Visualized Platform for Text Open Intent Recognitioncited by 24
  16. 2020Intentonomy: a Dataset and Study towards Human Intent Understandingcited by 22
  17. 2024Emotion and Intent Joint Understanding in Multimodal Conversation: A Benchmarking Datasetcited by 15
  18. 2021Knowledge Distillation from BERT Transformer to Speech Transformer for Intent Classificationcited by 14
  19. 2022Towards Textual Out-of-Domain Detection without In-Domain Labelscited by 14
  20. 2025Improving Generalization in Intent Detection: GRPO with Reward-Based Curriculum Samplingcited by 12
  21. 2019Multimodal Uncertainty Reduction for Intention Recognition in Human-Robot Interactioncited by 11
  22. 2022BERT Meets CTC: New Formulation of End-to-End Speech Recognition with Pre-trained Masked Language Modelcited by 11
  23. 2024LARA: Linguistic-Adaptive Retrieval-Augmentation for Multi-Turn Intent Classificationcited by 10
  24. 2024OmniActions: Predicting Digital Actions in Response to Real-World Multimodal Sensory Inputs with LLMscited by 10
  25. 2025A-MESS: Anchor based Multimodal Embedding with Semantic Synchronization for Multimodal Intent Recognitioncited by 10
  26. 2021Listen with Intent: Improving Speech Recognition with Audio-to-Intent Front-Endcited by 8
  27. 2023How to Communicate Robot Motion Intent: A Scoping Reviewcited by 8
  28. 2022Build-a-Bot: Teaching Conversational AI Using a Transformer-Based Intent Recognition and Question Answering Architecturecited by 7
  29. 2023SDIF-DA: A Shallow-to-Deep Interaction Framework with Data Augmentation for Multi-modal Intent Detectioncited by 7
  30. 2014Predicting Motivations of Actions by Leveraging Textcited by 6
  31. 2024Multimodal Classification and Out-of-distribution Detection for Multimodal Intent Understandingcited by 6
  32. 2024Dynamic Label Name Refinement for Few-Shot Dialogue Intent Classificationcited by 6
  33. 2025Knowledge-Decoupled Synergetic Learning: An MLLM based Collaborative Approach to Few-shot Multimodal Dialogue Intention Recognitioncited by 5
  34. 2022Audio-to-Intent Using Acoustic-Textual Subword Representations from End-to-End ASRcited by 4
  35. 2024Domain Adaptation in Intent Classification Systems: A Reviewcited by 4
  36. 2024Uddessho: An Extensive Benchmark Dataset for Multimodal Author Intent Classification in Low-Resource Bangla Languagecited by 2
  37. 2020Towards Better Graph Representation: Two-Branch Collaborative Graph Neural Networks for Multimodal Marketing Intention Detectioncited by 1
  38. 2023Generalized zero-shot audio-to-intent classificationcited by 1

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 5.

6 papers.

  1. 2025MVCL-DAF++: Enhancing Multimodal Intent Recognition via Prototype-Aware Contrastive Alignment and Coarse-to-Fine Dynamic Attention Fusioncited by 1
  2. 2026IntentVLM: Open-Vocabulary Intention Recognition through Forward-Inverse Modeling with Video-Language Modelscited by 0
  3. 2026Moltbook Moderation: Uncovering Hidden Intent Through Multi-Turn Dialoguecited by 0
  4. 2026Adaptive Modality Reliability Diagnosis and Restoration for Robust Multimodal Intent Recognitioncited by 0
  5. 2026Modality Agreement- and Conflict-Aware Prototype Hypergraph Learning for Multimodal Intent Understandingcited by 0
  6. 2026Neuro-Symbolic Graph Transformers for Hierarchical Intention Recognition from Multimodal Human Activity Sequencescited by 0