mlsanityData to 7 Oct 2026

Speech Emotion Recognition with Co-Attention based Multi-level Acoustic Information

arXiv 2203.15326 ↗First posted 29 Mar 2022cs.SD cs.AI eess.AScited by 31

Not on the map. Why?

Abstract (arXiv)

Speech Emotion Recognition (SER) aims to help the machine to understand human's subjective emotion from only audio information. However, extracting and utilizing comprehensive in-depth audio information is still a challenging task. In this paper, we propose an end-to-end speech emotion recognition system using multi-level acoustic information with a newly designed co-attention module. We firstly extract multi-level acoustic information, including MFCC, spectrogram, and the embedded high-level acoustic information with CNN, BiLSTM and wav2vec2, respectively. Then these extracted features are treated as multimodal inputs and fused by the proposed co-attention mechanism. Experiments are carried on the IEMOCAP dataset, and our model achieves competitive performance with two different speaker-independent cross-validation strategies. Our code is available on GitHub.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Builds on 5 of its 21 references that are in the map. First posted on arXiv 29 Mar 2022.

Builds on

5 of 21 references.

References · 5

  1. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  2. 2020Emotion recognition by fusing time synchronous and time asynchronous representationscited by 26
  3. 2021Efficient Speech Emotion Recognition Using Multi-Scale CNN and Attentioncited by 13
  4. 2019Attentive Modality Hopping Mechanism for Speech Emotion Recognitioncited by 6
  5. 2020Multi-Time-Scale Convolution for Emotion Recognition from Speech Audio Signalscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 11; 2024: 6; 2025: 5; 2026: 5.

31 papers.

Sort:

By area:

  1. 2023SpeechFormer++: A Hierarchical Efficient Framework for Paralinguistic Speech Processingcited by 16
  2. 2023Vesper: A Compact and Effective Pretrained Model for Speech Emotion Recognitioncited by 16
  3. 2023UniS-MMC: Multimodal Classification via Unimodality-supervised Multimodal Contrastive Learningcited by 13
  4. 2022Disentangling Prosody Representations with Unsupervised Speech Reconstructioncited by 8
  5. 2023DWFormer: Dynamic Window transFormer for Speech Emotion Recognitioncited by 7
  6. 2023Designing and Evaluating Speech Emotion Recognition Systems: A reality check case study with IEMOCAPcited by 6
  7. 2024MFHCA: Enhancing Speech Emotion Recognition Via Multi-Spatial Fusion and Hierarchical Cooperative Attentioncited by 6
  8. 2023Learning Robust Self-attention Features for Speech Emotion Recognition with Label-adaptive Mixupcited by 5
  9. 2022Knowledge Transfer For On-Device Speech Emotion Recognition with Neural Structured Learningcited by 4
  10. 2023Speech Emotion Recognition with Distilled Prosodic and Linguistic Affect Representationscited by 4
  11. 2023Test-Time Training for Speechcited by 4
  12. 2022Improving Speech Emotion Recognition with Unsupervised Speaking Style Transfercited by 3
19 more
  1. 2025Multimodal Large Language Models for End-to-End Affective Computing: Benchmarking and Boosting with Generative Knowledge Promptingcited by 3
  2. 2022Distribution-based Emotion Recognition in Conversationcited by 2
  3. 2024Exploring Multilingual Unseen Speaker Emotion Recognition: Leveraging Co-Attention Cues in Multitask Learningcited by 2
  4. 2024Two-stage Framework for Robust Speech Emotion Recognition Using Target Speaker Extraction in Human Speech Noise Conditionscited by 2
  5. 2026Complementarity-Supervised Spectral-Band Routing for Multimodal Emotion Recognitioncited by 2
  6. 2023Evaluating raw waveforms with deep learning frameworks for speech emotion recognitioncited by 1
  7. 2023A Multi-Task, Multi-Modal Approach for Predicting Categorical and Dimensional Emotionscited by 1
  8. 2025Qieemo: Speech Is All You Need in the Emotion Recognition in Conversationscited by 1
  9. 2025Balancing Task-invariant Interaction and Task-specific Adaptation for Unified Image Fusioncited by 1
  10. 2026PTS-SNN: A Prompt-Tuned Temporal Shift Spiking Neural Networks for Efficient Speech Emotion Recognitioncited by 1
  11. 2023MFSN: Multi-perspective Fusion Search Network For Pre-training Knowledge in Speech Emotion Recognitioncited by 0
  12. 2024CochCeps-Augment: A Novel Self-Supervised Contrastive Learning Using Cochlear Cepstrum-based Masking for Speech Emotion Recognitioncited by 0
  13. 2024Beyond the Labels: Unveiling Text-Dependency in Paralinguistic Speech Recognition Datasetscited by 0
  14. 2024emoDARTS: Joint Optimisation of CNN & Sequential Neural Network Architectures for Superior Speech Emotion Recognitioncited by 0
  15. 2025Large Language Models Meet Contrastive Learning: Zero-Shot Emotion Recognition Across Languagescited by 0
  16. 2025HarmoniFuse: A Component-Selective and Prompt-Adaptive Framework for Multi-Task Speech Language Modelingcited by 0
  17. 2026Uncertainty-Aware 3D Emotional Talking Face Synthesis with Emotion Prior Distillationcited by 0
  18. 2026Dual-Granularity Orthogonal Disentanglement for Generalizable Audio Deepfake Detectioncited by 0
  19. 2026Dual-Scale State-Space Modeling with Speaker-Wise Dynamic CRF for Speech Emotion Recognition in Conversationcited by 0