# A novel policy for pre-trained Deep Reinforcement Learning for Speech Emotion Recognition

arXiv: https://arxiv.org/abs/2101.00738 · HTML: https://mlsanity.com/paper/2101.00738/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 56 references that are in the map.
- First posted on arXiv 4 Jan 2021.

## Builds on (most cited first, 19 of 19)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [StarCraft II: A New Challenge for Reinforcement Learning](https://mlsanity.com/paper/1708.04782/) (2017)
- [A Theoretical Analysis of Deep Q-Learning](https://mlsanity.com/paper/1901.00137/) (2019)
- [Boltzmann Exploration Done Right](https://mlsanity.com/paper/1705.10257/) (2017)
- [Deep Representation Learning in Speech Processing: Challenges, Recent Advances, and Future Trends](https://mlsanity.com/paper/2001.00378/) (2020)
- [Multi-Modal Emotion recognition on IEMOCAP Dataset using Deep Learning](https://mlsanity.com/paper/1804.05788/) (2018)
- [CNN+LSTM Architecture for Speech Emotion Recognition with Data Augmentation](https://mlsanity.com/paper/1802.05630/) (2018)
- [Pre-training Neural Networks with Human Demonstrations for Deep Reinforcement Learning](https://mlsanity.com/paper/1709.04083/) (2017)
- [The Atari Grand Challenge Dataset](https://mlsanity.com/paper/1705.10998/) (2017)
- [Reinforcement Learning with Dynamic Boltzmann Softmax Updates](https://mlsanity.com/paper/1903.05926/) (2019)
- [Direct Modelling of Speech Emotion from Raw Speech](https://mlsanity.com/paper/1904.03833/) (2019)
- [Reinforcement Learning Based Speech Enhancement for Robust Speech Recognition](https://mlsanity.com/paper/1811.04224/) (2018)
- [Adversarial Machine Learning And Speech Emotion Recognition: Utilizing Generative Adversarial Networks For Robustness](https://mlsanity.com/paper/1811.11402/) (2018)
- [EmoRL: Continuous Acoustic Emotion Classification using Deep Reinforcement Learning](https://mlsanity.com/paper/1804.04053/) (2018)
- [Reinforcement Learning To Adapt Speech Enhancement to Instantaneous Input Signal Quality](https://mlsanity.com/paper/1711.10791/) (2017)
- [Model-Based Regularization for Deep Reinforcement Learning with Transcoder Networks](https://mlsanity.com/paper/1809.01906/) (2018)
- [Automated Screening for Distress: A Perspective for the Future](https://mlsanity.com/paper/1902.09944/) (2019)
- [Pre-training in Deep Reinforcement Learning for Automatic Speech Recognition](https://mlsanity.com/paper/1910.11256/) (2019)
- [A breakthrough in Speech emotion recognition using Deep Retinal Convolution Neural Networks](https://mlsanity.com/paper/1707.09917/) (2017)

## Built on by (most cited first, 2 of 2)

- [Domain Adapting Deep Reinforcement Learning for Real-world Speech Emotion Recognition](https://mlsanity.com/paper/2207.12248/) (2022)
- [UF-AMA: A unified framework for cross-domain emotion recognition via adaptive multimodal alignment](https://mlsanity.com/paper/2606.00170/) (2026)

## Abstract (arXiv)

Reinforcement Learning (RL) is a semi-supervised learning paradigm which an agent learns by interacting with an environment. Deep learning in combination with RL provides an efficient method to learn how to interact with the environment is called Deep Reinforcement Learning (deep RL). Deep RL has gained tremendous success in gaming - such as AlphaGo, but its potential have rarely being explored for challenging tasks like Speech Emotion Recognition (SER). The deep RL being used for SER can potentially improve the performance of an automated call centre agent by dynamically learning emotional-aware response to customer queries. While the policy employed by the RL agent plays a major role in action selection, there is no current RL policy tailored for SER. In addition, extended learning period is a general challenge for deep RL which can impact the speed of learning for SER. Therefore, in this paper, we introduce a novel policy - "Zeta policy" which is tailored for SER and apply Pre-training in deep RL to achieve faster learning rate. Pre-training with cross dataset was also studied to discover the feasibility of pre-training the RL Agent with a similar dataset in a scenario of where no real environmental data is not available. IEMOCAP and SAVEE datasets were used for the evaluation with the problem being to recognize four emotions happy, sad, angry and neutral in the utterances provided. Experimental results show that the proposed "Zeta policy" performs better than existing policies. The results also support that pre-training can reduce the training time upon reducing the warm-up period and is robust to cross-corpus scenario.