# Text Takes Over: A Study of Modality Bias in Multimodal Intent Detection

arXiv: https://arxiv.org/abs/2508.16122 · HTML: https://mlsanity.com/paper/2508.16122/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 53 references that are in the map.
- First posted on arXiv 22 Aug 2025.

## Builds on (most cited first, 30 of 30)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding](https://mlsanity.com/paper/2306.02858/) (2023)
- [Video-LLaVA: Learning United Visual Representation by Alignment Before Projection](https://mlsanity.com/paper/2311.10122/) (2023)
- [Multimodal Transformer for Unaligned Multimodal Language Sequences](https://mlsanity.com/paper/1906.00295/) (2019)
- [Efficient Intent Detection with Dual Sentence Encoders](https://mlsanity.com/paper/2003.04807/) (2020)
- [Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces](https://mlsanity.com/paper/1805.10190/) (2018)
- [MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2005.03545/) (2020)
- [Zero-shot User Intent Detection via Capsule Neural Networks](https://mlsanity.com/paper/1809.00385/) (2018)
- [MIntRec: A New Dataset for Multimodal Intent Recognition](https://mlsanity.com/paper/2209.04355/) (2022)
- [Integrating Text and Image: Determining Multimodal Document Intent in Instagram Posts](https://mlsanity.com/paper/1904.09073/) (2019)
- [Incremental Few-shot Text Classification with Multi-round New Classes: Formulation, Dataset and System](https://mlsanity.com/paper/2104.11882/) (2021)
- [MIntRec2.0: A Large-scale Benchmark Dataset for Multimodal Intent Recognition and Out-of-scope Detection in Conversations](https://mlsanity.com/paper/2403.10943/) (2024)
- [On Modality Bias Recognition and Reduction](https://mlsanity.com/paper/2202.12690/) (2022)
- [CrysMMNet: Multimodal Representation for Crystal Property Prediction](https://mlsanity.com/paper/2307.05390/) (2023)
- [Classification under Streaming Emerging New Classes: A Solution using Completely Random Trees](https://mlsanity.com/paper/1605.09131/) (2016)
- [Periodic Materials Generation using Text-Guided Joint Diffusion Model](https://mlsanity.com/paper/2503.00522/) (2025)
- [MatSciRE: Leveraging Pointer Networks to Automate Entity and Relation Extraction for Material Science Knowledge-base Construction](https://mlsanity.com/paper/2401.09839/) (2024)
- [MatScIE: An automated tool for the generation of databases of methods and parameters used in the computational materials science literature](https://mlsanity.com/paper/2009.06819/) (2020)
- [An Evaluation Framework for Legal Document Summarization](https://mlsanity.com/paper/2205.08478/) (2022)
- [Intent Identification and Entity Extraction for Healthcare Queries in Indic Languages](https://mlsanity.com/paper/2302.09685/) (2023)
- [Fine-grained Intent Classification in the Legal Domain](https://mlsanity.com/paper/2205.03509/) (2022)
- [SDIF-DA: A Shallow-to-Deep Interaction Framework with Data Augmentation for Multi-modal Intent Detection](https://mlsanity.com/paper/2401.00424/) (2023)
- [Novel Intent Detection and Active Learning Based Classification (Student Abstract)](https://mlsanity.com/paper/2304.11058/) (2023)
- [Intent Detection and Entity Extraction from BioMedical Literature](https://mlsanity.com/paper/2404.03598/) (2024)
- [RTE: A Tool for Annotating Relation Triplets from Text](https://mlsanity.com/paper/2108.08184/) (2021)
- [A Pointer Network-based Approach for Joint Extraction and Detection of Multi-Label Multi-Class Intents](https://mlsanity.com/paper/2410.22476/) (2024)

## Built on by (most cited first, 7 of 7)

- [CAAD: Contrastive Audio-Aware Distillation for Efficient Speech Language Models](https://mlsanity.com/paper/2606.23052/) (2026)
- [Physics-based phenomenological characterization of cross-modal bias in multimodal models](https://mlsanity.com/paper/2602.20624/) (2026)
- [Omni Demand Understanding: A Benchmark for Contextual User-Intent Inference in Multimodal Interaction](https://mlsanity.com/paper/2609.21392/) (2026)
- [IDALC: A Semi-Supervised Framework for Intent Detection and Active Learning based Correction](https://mlsanity.com/paper/2511.05921/) (2025)
- [DISC: Decoupling Instruction from State-Conditioned Control via Policy Generation](https://mlsanity.com/paper/2605.20856/) (2026)
- [Adaptive Modality Reliability Diagnosis and Restoration for Robust Multimodal Intent Recognition](https://mlsanity.com/paper/2608.03475/) (2026)
- [EgoArgus: Benchmarking VLMs as Situational Assistants for Modality-Grounded User Supports](https://mlsanity.com/paper/2608.25561/) (2026)

## Abstract (arXiv)

The rise of multimodal data, integrating text, audio, and visuals, has created new opportunities for studying multimodal tasks such as intent detection. This work investigates the effectiveness of Large Language Models (LLMs) and non-LLMs, including text-only and multi-modal models, in the multimodal intent detection task. Our study reveals that Mistral-7B, a text-only LLM, outperforms most competitive multimodal models by approximately 9% on MIntRec-1 and 4% on MIntRec2.0 datasets. This performance advantage comes from a strong textual bias in these datasets, where over 90% of the samples require textual input, either alone or in combination with other modalities, for correct classification. We confirm the modality bias of these datasets via human evaluation, too. Next, we propose a framework to debias the datasets, and upon debiasing, more than 70% of the samples in MIntRec-1 and more than 50% in MIntRec2.0 get removed, resulting in significant performance degradation across all models, with smaller multimodal fusion models being the most affected with an accuracy drop of over 50 - 60%. Further, we analyze the context-specific relevance of different modalities through empirical analysis. Our findings highlight the challenges posed by modality bias in multimodal intent datasets and emphasize the need for unbiased datasets to evaluate multimodal models effectively.