# MFSN: Multi-perspective Fusion Search Network For Pre-training Knowledge in Speech Emotion Recognition

arXiv: https://arxiv.org/abs/2306.09361 · HTML: https://mlsanity.com/paper/2306.09361/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 24 references that are in the map.
- First posted on arXiv 12 Jun 2023.

## Builds on (most cited first, 13 of 13)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language](https://mlsanity.com/paper/2202.03555/) (2022)
- [vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations](https://mlsanity.com/paper/1910.05453/) (2019)
- [TERA: Self-Supervised Learning of Transformer Encoder Representation for Speech](https://mlsanity.com/paper/2007.06028/) (2020)
- [Emotion Recognition from Speech Using Wav2vec 2.0 Embeddings](https://mlsanity.com/paper/2104.03502/) (2021)
- [A Fine-tuned Wav2vec 2.0/HuBERT Benchmark For Speech Emotion Recognition, Speaker Verification and Spoken Language Understanding](https://mlsanity.com/paper/2111.02735/) (2021)
- [Speech Emotion Recognition with Co-Attention based Multi-level Acoustic Information](https://mlsanity.com/paper/2203.15326/) (2022)
- [Emotion recognition by fusing time synchronous and time asynchronous representations](https://mlsanity.com/paper/2010.14102/) (2020)
- [Learning Fine-Grained Cross Modality Excitement for Speech Emotion Recognition](https://mlsanity.com/paper/2010.12733/) (2020)
- [CTL-MTNet: A Novel CapsNet and Transfer Learning-Based Mixed Task Net for the Single-Corpus and Cross-Corpus Speech Emotion Recognition](https://mlsanity.com/paper/2207.10644/) (2022)
- [Disentangling Prosody Representations with Unsupervised Speech Reconstruction](https://mlsanity.com/paper/2212.06972/) (2022)

## Abstract (arXiv)

Speech Emotion Recognition (SER) is an important research topic in human-computer interaction. Many recent works focus on directly extracting emotional cues through pre-trained knowledge, frequently overlooking considerations of appropriateness and comprehensiveness. Therefore, we propose a novel framework for pre-training knowledge in SER, called Multi-perspective Fusion Search Network (MFSN). Considering comprehensiveness, we partition speech knowledge into Textual-related Emotional Content (TEC) and Speech-related Emotional Content (SEC), capturing cues from both semantic and acoustic perspectives, and we design a new architecture search space to fully leverage them. Considering appropriateness, we verify the efficacy of different modeling approaches in capturing SEC and fills the gap in current research. Experimental results on multiple datasets demonstrate the superiority of MFSN.