mlsanityData to 14 Oct 2026

Joint Feature and Output Distillation for Low-complexity Acoustic Scene Classification

arXiv 2507.19557 ↗First posted 25 Jul 2025cs.SD eess.AScited by 0

On the map: Knowledge distillation → Speech and audio distillation

Abstract (arXiv)

This report presents a dual-level knowledge distillation framework with multi-teacher guidance for low-complexity acoustic scene classification (ASC) in DCASE2025 Task 1. We propose a distillation strategy that jointly transfers both soft logits and intermediate feature representations. Specifically, we pre-trained PaSST and CP-ResNet models as teacher models. Logits from teachers are averaged to generate soft targets, while one CP-ResNet is selected for feature-level distillation. This enables the compact student model (CP-Mobile) to capture both semantic distribution and structural information from teacher guidance. Experiments on the TAU Urban Acoustic Scenes 2022 Mobile dataset (development set) demonstrate that our submitted systems achieve up to 59.30% accuracy.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Speech and audio distillation, one of 12 directions in Knowledge distillation. Builds on 8 of its 18 references that are in the map. First posted on arXiv 25 Jul 2025.

Builds on

8 of 18 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Data Efficient Acoustic Scene Classification using Teacher-Informed Confusing Class Instruction18 Sep
  2. 2023Two-Step Knowledge Distillation for Tiny Speech Enhancement15 Sep

Other references · 6

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  2. 2014FitNets: Hints for Thin Deep Netscited by 1,653
  3. 2020Knowledge Distillation: A Surveycited by 1,603
  4. 2021Domain Generalization with MixStylecited by 566
  5. 2021Efficient Training of Audio Transformers with Patchoutcited by 226
  6. 2019Receptive-field-regularized CNN variants for acoustic scene classificationcited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Data-Efficient Low-Complexity Acoustic Scene Classification via Distilling and Progressive Pruning
  2. 2025Adaptive Knowledge Distillation using a Device-Aware Teacher for Low-Complexity Acoustic Scene Classification
  3. 2025Creating a Good Teacher for Knowledge Distillation in Acoustic Scene Classification
  4. 2023Low-complexity deep learning frameworks for acoustic scene classification using teacher-student scheme and multiple spectrograms
  5. 2025Acoustic Scene Classification Using CNN-GRU Model Without Knowledge Distillation
  6. 2025Improving Acoustic Scene Classification with City Features
  7. 2022Low-complexity CNNs for Acoustic Scene Classification
  8. 2024Improving Acoustic Scene Classification in Low-Resource Conditions