# emoDARTS: Joint Optimisation of CNN & Sequential Neural Network Architectures for Superior Speech Emotion Recognition

arXiv: https://arxiv.org/abs/2403.14083 · HTML: https://mlsanity.com/paper/2403.14083/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 43 references that are in the map.
- First posted on arXiv 21 Mar 2024.

## Builds on (most cited first, 19 of 19)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [Progressive Neural Architecture Search](https://mlsanity.com/paper/1712.00559/) (2017)
- [PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Search](https://mlsanity.com/paper/1907.05737/) (2019)
- [FBNetV2: Differentiable Neural Architecture Search for Spatial and Channel Dimensions](https://mlsanity.com/paper/2004.05565/) (2020)
- [DARTS+: Improved Differentiable Architecture Search with Early Stopping](https://mlsanity.com/paper/1909.06035/) (2019)
- [Stabilizing Differentiable Architecture Search via Perturbation-based Regularization](https://mlsanity.com/paper/2002.05283/) (2020)
- [μNAS: Constrained Neural Architecture Search for Microcontrollers](https://mlsanity.com/paper/2010.14246/) (2020)
- [Speech Emotion Recognition with Co-Attention based Multi-level Acoustic Information](https://mlsanity.com/paper/2203.15326/) (2022)
- [Attention Based Fully Convolutional Network for Speech Emotion Recognition](https://mlsanity.com/paper/1806.01506/) (2018)
- [Cyclic Differentiable Architecture Search](https://mlsanity.com/paper/2006.10724/) (2020)
- [Direct Modelling of Speech Emotion from Raw Speech](https://mlsanity.com/paper/1904.03833/) (2019)
- [Self Supervised Adversarial Domain Adaptation for Cross-Corpus and Cross-Language Speech Emotion Recognition](https://mlsanity.com/paper/2204.08625/) (2022)
- [Multitask Learning from Augmented Auxiliary Data for Improving Speech Emotion Recognition](https://mlsanity.com/paper/2207.05298/) (2022)
- [Neural Architecture Search for Speech Emotion Recognition](https://mlsanity.com/paper/2203.16928/) (2022)
- [Image denoising and restoration with CNN-LSTM Encoder Decoder with Direct Attention](https://mlsanity.com/paper/1801.05141/) (2018)
- [EmotionNAS: Two-stream Neural Architecture Search for Speech Emotion Recognition](https://mlsanity.com/paper/2203.13617/) (2022)
- [Enhancing Speech Emotion Recognition Through Differentiable Architecture Search](https://mlsanity.com/paper/2305.14402/) (2023)

## Abstract (arXiv)

Speech Emotion Recognition (SER) is crucial for enabling computers to understand the emotions conveyed in human communication. With recent advancements in Deep Learning (DL), the performance of SER models has significantly improved. However, designing an optimal DL architecture requires specialised knowledge and experimental assessments. Fortunately, Neural Architecture Search (NAS) provides a potential solution for automatically determining the best DL model. The Differentiable Architecture Search (DARTS) is a particularly efficient method for discovering optimal models. This study presents emoDARTS, a DARTS-optimised joint CNN and Sequential Neural Network (SeqNN: LSTM, RNN) architecture that enhances SER performance. The literature supports the selection of CNN and LSTM coupling to improve performance. While DARTS has previously been used to choose CNN and LSTM operations independently, our technique adds a novel mechanism for selecting CNN and SeqNN operations in conjunction using DARTS. Unlike earlier work, we do not impose limits on the layer order of the CNN. Instead, we let DARTS choose the best layer order inside the DARTS cell. We demonstrate that emoDARTS outperforms conventionally designed CNN-LSTM models and surpasses the best-reported SER results achieved through DARTS on CNN-LSTM by evaluating our approach on the IEMOCAP, MSP-IMPROV, and MSP-Podcast datasets.