mlsanityData to 7 Oct 2026

A-MESS: Anchor based Multimodal Embedding with Semantic Synchronization for Multimodal Intent Recognition

arXiv 2503.19474 ↗First posted 25 Mar 2025cs.CV cs.AIcited by 10

Not on the map. Why?

Abstract (arXiv)

In the domain of multimodal intent recognition (MIR), the objective is to recognize human intent by integrating a variety of modalities, such as language text, body gestures, and tones. However, existing approaches face difficulties adequately capturing the intrinsic connections between the modalities and overlooking the corresponding semantic representations of intent. To address these limitations, we present the Anchor-based Multimodal Embedding with Semantic Synchronization (A-MESS) framework. We first design an Anchor-based Multimodal Embedding (A-ME) module that employs an anchor-based embedding fusion mechanism to integrate multimodal inputs. Furthermore, we develop a Semantic Synchronization (SS) strategy with the Triplet Contrastive Learning pipeline, which optimizes the process by synchronizing multimodal representation with label descriptions produced by the large language model. Comprehensive experiments indicate that our A-MESS achieves state-of-the-art and provides substantial insight into multimodal representation and downstream tasks.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 12 of its 15 references that are in the map. First posted on arXiv 25 Mar 2025.

Builds on

12 of 15 references.

References · 12

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  5. 2023GPT-4 Technical Reportcited by 18,132
  6. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  7. 2017Decoupled Weight Decay Regularizationcited by 1,722
  8. 2020TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialoguecited by 185
  9. 2024Agent AI: Surveying the Horizons of Multimodal Interactioncited by 167
  10. 2022MIntRec: A New Dataset for Multimodal Intent Recognitioncited by 41
  11. 2024MIntRec2.0: A Large-scale Benchmark Dataset for Multimodal Intent Recognition and Out-of-scope Detection in Conversationscited by 25
  12. 2023Token-Level Contrastive Learning with Modality-Aware Prompting for Multimodal Intent Recognitioncited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 7; 2026: 3.

10 papers.

  1. 2025Deep Learning Approaches for Multimodal Intent Recognition: A Surveycited by 5
  2. 2025Butter: Frequency Consistency and Hierarchical Fusion for Autonomous Driving Object Detectioncited by 4
  3. 2026GraphiContact: Pose-aware Human-Scene Robust Contact Perception for Interactive Systemscited by 1
  4. 2026Editing on the Generative Manifold: A Theoretical and Empirical Study of General Diffusion-Based Image Editing Trade-offscited by 1
  5. 2025FADE: Adversarial Concept Erasure in Flow Modelscited by 0
  6. 2025Multi-Modal Machine Learning Framework for Predicting Early Recurrence of Brain Tumors Using MRI and Clinical Biomarkerscited by 0
  7. 2025A Multimodal Deep Learning Framework for Early Diagnosis of Liver Cancer via Optimized BiLSTM-AM-VMD Architecturecited by 0
  8. 2025Robust Concept Erasure in Diffusion Models: A Theoretical Perspective on Security and Robustnesscited by 0
  9. 2025DialogGraph-LLM: Graph-Informed LLMs for End-to-End Audio Dialogue Intent Recognitioncited by 0
  10. 2026Leveraging Turn-taking Dynamics for Intent Recognition in Multi-party Conversationscited by 0