# Low-complexity deep learning frameworks for acoustic scene classification using teacher-student scheme and multiple spectrograms

arXiv: https://arxiv.org/abs/2305.09463 · HTML: https://mlsanity.com/paper/2305.09463/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Speech and audio distillation, one of 12 directions in Knowledge distillation.
- Builds on 16 of its 26 references that are in the map.
- First posted on arXiv 16 May 2023.

## Placed in

- Knowledge distillation / [Speech and audio distillation](https://mlsanity.com/distill/speech-and-audio-distillation/)

## Builds on (most cited first, 16 of 16)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [Data Augmentation using Random Image Cropping and Patching for Deep CNNs](https://mlsanity.com/paper/1811.09030/) (2018)
- [Learning from Between-class Examples for Deep Sound Recognition](https://mlsanity.com/paper/1711.10282/) (2017)
- [Mixup-Based Acoustic Scene Classification Using Multi-Channel Convolutional Neural Network](https://mlsanity.com/paper/1805.07319/) (2018)
- [SubSpectralNet - Using Sub-Spectrogram based Convolutional Neural Networks for Acoustic Scene Classification](https://mlsanity.com/paper/1810.12642/) (2018)
- [Robust Acoustic Scene Classification using a Multi-Spectrogram Encoder-Decoder Framework](https://mlsanity.com/paper/2002.04502/) (2020)
- [Multi-view Audio and Music Classification](https://mlsanity.com/paper/2103.02420/) (2021)
- [Wider or Deeper Neural Network Architecture for Acoustic Scene Classification with Mismatched Recording Devices](https://mlsanity.com/paper/2203.12314/) (2022)
- [Spatio-Temporal Attention Pooling for Audio Scene Classification](https://mlsanity.com/paper/1904.03543/) (2019)
- [Acoustic Scene Classification by Implicitly Identifying Distinct Sound Events](https://mlsanity.com/paper/1904.05204/) (2019)
- [Deep Feature Embedding and Hierarchical Classification for Audio Scene Classification](https://mlsanity.com/paper/2002.04857/) (2020)
- [Sound Context Classification Basing on Join Learning Model and Multi-Spectrogram Features](https://mlsanity.com/paper/2005.12779/) (2020)
- [Deep Learning Frameworks Applied For Audio-Visual Scene Classification](https://mlsanity.com/paper/2106.06840/) (2021)
- [A Low-Compexity Deep Learning Framework For Acoustic Scene Classification](https://mlsanity.com/paper/2106.06838/) (2021)
- [Robust, General, and Low Complexity Acoustic Scene Classification Systems and An Effective Visualization for Presenting a Sound Scene Context](https://mlsanity.com/paper/2210.08610/) (2022)

## Abstract (arXiv)

In this technical report, a low-complexity deep learning system for acoustic scene classification (ASC) is presented. The proposed system comprises two main phases: (Phase I) Training a teacher network; and (Phase II) training a student network using distilled knowledge from the teacher. In the first phase, the teacher, which presents a large footprint model, is trained. After training the teacher, the embeddings, which are the feature map of the second last layer of the teacher, are extracted. In the second phase, the student network, which presents a low complexity model, is trained with the embeddings extracted from the teacher. Our experiments conducted on DCASE 2023 Task 1 Development dataset have fulfilled the requirement of low-complexity and achieved the best classification accuracy of 57.4%, improving DCASE baseline by 14.5%.