mlsanityData to 14 Oct 2026

DyKen-Hyena: Dynamic Kernel Generation via Cross-Modal Attention for Multimodal Intent Recognition

arXiv 2509.09940 ↗First posted 12 Sep 2025cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Though Multimodal Intent Recognition (MIR) proves effective by utilizing rich information from multiple sources (e.g., language, video, and audio), the potential for intent-irrelevant and conflicting information across modalities may hinder performance from being further improved. Most current models attempt to fuse modalities by applying mechanisms like multi-head attention to unimodal feature sequences and then adding the result back to the original representation. This process risks corrupting the primary linguistic features with noisy or irrelevant non-verbal signals, as it often fails to capture the fine-grained, token-level influence where non-verbal cues should modulate, not just augment, textual meaning. To address this, we introduce DyKen-Hyena, which reframes the problem from feature fusion to processing modulation. Our model translates audio-visual cues into dynamic, per-token convolutional kernels that directly modulate textual feature extraction. This fine-grained approach achieves state-of-the-art results on the MIntRec and MIntRec2.0 benchmarks. Notably, it yields a +10.46% F1-score improvement in out-of-scope detection, validating that our method creates a fundamentally more robust intent representation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 22 references that are in the map. First posted on arXiv 12 Sep 2025.

Builds on

16 of 22 references.

References · 16

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  4. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
  5. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,553
  6. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,214
  7. 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,013
  8. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 727
  9. 2017Tensor Fusion Network for Multimodal Sentiment Analysiscited by 433
  10. 2023Hyena Hierarchy: Towards Larger Convolutional Language Modelscited by 393
  11. 2016Dynamic Filter Networkscited by 319
  12. 2018Efficient Low-rank Multimodal Fusion with Modality-Specific Factorscited by 270
4 more
  1. 2020Beneath the Tip of the Iceberg: Current Challenges and New Directions in Sentiment Analysis Researchcited by 67
  2. 2022MIntRec: A New Dataset for Multimodal Intent Recognitioncited by 41
  3. 2024MIntRec2.0: A Large-scale Benchmark Dataset for Multimodal Intent Recognition and Out-of-scope Detection in Conversationscited by 26
  4. 2018Deep Multimodal Learning for Emotion Recognition in Spoken Languagecited by 3

Built on by

Not cited yet.