# Environmental sound analysis with mixup based multitask learning and cross-task fusion

arXiv: https://arxiv.org/abs/2103.16079 · HTML: https://mlsanity.com/paper/2103.16079/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 23 references that are in the map.
- First posted on arXiv 30 Mar 2021.

## Builds on (most cited first, 7 of 7)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Cross-stitch Networks for Multi-task Learning](https://mlsanity.com/paper/1604.03539/) (2016)
- [Knowledge Transfer from Weakly Labeled Audio using Convolutional Neural Network for Sound Events and Scenes](https://mlsanity.com/paper/1711.01369/) (2017)
- [Towards joint sound scene and polyphonic sound event recognition](https://mlsanity.com/paper/1904.10408/) (2019)
- [Sound Event Detection by Multitask Learning of Sound Events and Scenes with Soft Scene Labels](https://mlsanity.com/paper/2002.05848/) (2020)
- [Joint Analysis of Acoustic Events and Scenes Based on Multitask Learning](https://mlsanity.com/paper/1904.12146/) (2019)

## Abstract (arXiv)

Environmental sound analysis is currently getting more and more attentions. In the domain, acoustic scene classification and acoustic event classification are two closely related tasks. In this letter, a two-stage method is proposed for the above tasks. In the first stage, a mixup based MTL solution is proposed to classify both tasks in one single convolutional neural network. Artificial multi-label samples are used in the training of the MTL model, which are mixed up using existing single-task datasets. The multi-task model obtained can effectively recognize both the acoustic scenes and events. Compared with other methods such as re-annotation or synthesis, the mixup based MTL is low-cost, flexible and effective. In the second stage, the MTL model is modified into a single-task model which is fine-tuned using the original dataset corresponding to the specific task. By controlling the frozen layers carefully, the task-specific high level features are fused and the performance of the single classification task is further improved. The proposed method has confirmed the complementary characteristics of acoustic scene and acoustic event classifications. Finally, enhanced by ensemble learning, a satisfactory accuracy of 84.5 percent on TUT acoustic scene 2017 dataset and an accuracy of 77.5 percent on ESC-50 dataset are achieved respectively.