# CTL-MTNet: A Novel CapsNet and Transfer Learning-Based Mixed Task Net for the Single-Corpus and Cross-Corpus Speech Emotion Recognition

arXiv: https://arxiv.org/abs/2207.10644 · HTML: https://mlsanity.com/paper/2207.10644/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 37 references that are in the map.
- First posted on arXiv 18 Jul 2022.

## Builds on (most cited first, 7 of 7)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017)
- [Bridging Theory and Algorithm for Domain Adaptation](https://mlsanity.com/paper/1904.05801/) (2019)
- [Seen and Unseen emotional style transfer for voice conversion with a new emotional speech dataset](https://mlsanity.com/paper/2010.14794/) (2020)
- [ERANNs: Efficient Residual Audio Neural Networks for Audio Pattern Recognition](https://mlsanity.com/paper/2106.01621/) (2021)
- [Domain Adversarial for Acoustic Emotion Recognition](https://mlsanity.com/paper/1804.07690/) (2018)

## Built on by (most cited first, 8 of 8)

- [Temporal Modeling Matters: A Novel Temporal Emotional Modeling Approach for Speech Emotion Recognition](https://mlsanity.com/paper/2211.08233/) (2022)
- [Emo-DNA: Emotion Decoupling and Alignment Learning for Cross-Corpus Speech Emotion Recognition](https://mlsanity.com/paper/2308.02190/) (2023)
- [MSAC: Multiple Speech Attribute Control Method for Reliable Speech Emotion Recognition](https://mlsanity.com/paper/2308.04025/) (2023)
- [SPEAKER VGG CCT: Cross-corpus Speech Emotion Recognition with Speaker Embedding and Vision Transformers](https://mlsanity.com/paper/2211.02366/) (2022)
- [MFSN: Multi-perspective Fusion Search Network For Pre-training Knowledge in Speech Emotion Recognition](https://mlsanity.com/paper/2306.09361/) (2023)
- [Active Learning with Task Adaptation Pre-training for Speech Emotion Recognition](https://mlsanity.com/paper/2405.00307/) (2024)
- [EmoHRNet: High-Resolution Neural Network Based Speech Emotion Recognition](https://mlsanity.com/paper/2510.06072/) (2025)
- [Hierarchical Codec Diffusion for Video-to-Speech Generation](https://mlsanity.com/paper/2604.15923/) (2026)

## Abstract (arXiv)

Speech Emotion Recognition (SER) has become a growing focus of research in human-computer interaction. An essential challenge in SER is to extract common attributes from different speakers or languages, especially when a specific source corpus has to be trained to recognize the unknown data coming from another speech corpus. To address this challenge, a Capsule Network (CapsNet) and Transfer Learning based Mixed Task Net (CTLMTNet) are proposed to deal with both the singlecorpus and cross-corpus SER tasks simultaneously in this paper. For the single-corpus task, the combination of Convolution-Pooling and Attention CapsNet module CPAC) is designed by embedding the self-attention mechanism to the CapsNet, guiding the module to focus on the important features that can be fed into different capsules. The extracted high-level features by CPAC provide sufficient discriminative ability. Furthermore, to handle the cross-corpus task, CTL-MTNet employs a Corpus Adaptation Adversarial Module (CAAM) by combining CPAC with Margin Disparity Discrepancy (MDD), which can learn the domain-invariant emotion representations through extracting the strong emotion commonness. Experiments including ablation studies and visualizations on both singleand cross-corpus tasks using four well-known SER datasets in different languages are conducted for performance evaluation and comparison. The results indicate that in both tasks the CTL-MTNet showed better performance in all cases compared to a number of state-of-the-art methods. The source code and the supplementary materials are available at: https://github.com/MLDMXM2017/CTLMTNet