# Rethinking CNN Models for Audio Classification

arXiv: https://arxiv.org/abs/2007.11154 · HTML: https://mlsanity.com/paper/2007.11154/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Jul 2020.

## Built on by (most cited first, 30 of 48)

- [AST: Audio Spectrogram Transformer](https://mlsanity.com/paper/2104.01778/) (2021)
- [LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment](https://mlsanity.com/paper/2310.01852/) (2023)
- [AudioCLIP: Extending CLIP to Image, Text and Audio](https://mlsanity.com/paper/2106.13043/) (2021)
- [PSLA: Improving Audio Tagging with Pretraining, Sampling, Labeling, and Aggregation](https://mlsanity.com/paper/2102.01243/) (2021)
- [Emotional Speech-driven 3D Body Animation via Disentangled Latent Diffusion](https://mlsanity.com/paper/2312.04466/) (2023)
- [Auto-ACD: A Large-scale Dataset for Audio-Language Representation Learning](https://mlsanity.com/paper/2309.11500/) (2023)
- [Single-Layer Vision Transformers for More Accurate Early Exits with Less Overhead](https://mlsanity.com/paper/2105.09121/) (2021)
- [Exploring Vision Transformers for 3D Human Motion-Language Models with Motion Patches](https://mlsanity.com/paper/2405.04771/) (2024)
- [Images that Sound: Composing Images and Sounds on a Single Canvas](https://mlsanity.com/paper/2405.12221/) (2024)
- [Domain Generalization with Relaxed Instance Frequency-wise Normalization for Multi-device Acoustic Scene Classification](https://mlsanity.com/paper/2206.12513/) (2022)
- [HumBugDB: A Large-scale Acoustic Mosquito Dataset](https://mlsanity.com/paper/2110.07607/) (2021)
- [Continual Transformers: Redundancy-Free Attention for Online Inference](https://mlsanity.com/paper/2201.06268/) (2022)
- [CMKD: CNN/Transformer-Based Cross-Model Knowledge Distillation for Audio Classification](https://mlsanity.com/paper/2203.06760/) (2022)
- [Investigating Multi-Feature Selection and Ensembling for Audio Classification](https://mlsanity.com/paper/2206.07511/) (2022)
- [SemanticAC: Semantics-Assisted Framework for Audio Classification](https://mlsanity.com/paper/2302.05940/) (2023)
- [Detection of Audio-Video Synchronization Errors Via Event Detection](https://mlsanity.com/paper/2104.10116/) (2021)
- [Dual Bayesian ResNet: A Deep Learning Approach to Heart Murmur Detection](https://mlsanity.com/paper/2305.16691/) (2023)
- [FastAST: Accelerating Audio Spectrogram Transformer via Token Merging and Cross-Model Knowledge Distillation](https://mlsanity.com/paper/2406.07676/) (2024)
- [Imperceptible Rhythm Backdoor Attacks: Exploring Rhythm Transformation for Embedding Undetectable Vulnerabilities on Speech Recognition](https://mlsanity.com/paper/2406.10932/) (2024)
- [SoundCLR: Contrastive Learning of Representations For Improved Environmental Sound Classification](https://mlsanity.com/paper/2103.01929/) (2021)
- [QbyE-MLPMixer: Query-by-Example Open-Vocabulary Keyword Spotting using MLPMixer](https://mlsanity.com/paper/2206.13231/) (2022)
- [Towards Controllable Audio Texture Morphing](https://mlsanity.com/paper/2304.11648/) (2023)
- [AudRandAug: Random Image Augmentations for Audio Classification](https://mlsanity.com/paper/2309.04762/) (2023)
- [Towards generalizing deep-audio fake detection networks](https://mlsanity.com/paper/2305.13033/) (2023)
- [Continual Low-Rank Scaled Dot-product Attention](https://mlsanity.com/paper/2412.03214/) (2024)
- [Do sound event representations generalize to other audio tasks? A case study in audio transfer learning](https://mlsanity.com/paper/2106.11335/) (2021)
- [A Visual Domain Transfer Learning Approach for Heartbeat Sound Classification](https://mlsanity.com/paper/2107.13237/) (2021)
- [NeuroView: Explainable Deep Network Decision Making](https://mlsanity.com/paper/2110.07778/) (2021)
- [UFRC: A Unified Framework for Reliable COVID-19 Detection on Crowdsourced Cough Audio](https://mlsanity.com/paper/2204.07763/) (2022)
- [Example-Based Framework for Perceptually Guided Audio Texture Generation](https://mlsanity.com/paper/2308.11859/) (2023)

## Abstract (arXiv)

In this paper, we show that ImageNet-Pretrained standard deep CNN models can be used as strong baseline networks for audio classification. Even though there is a significant difference between audio Spectrogram and standard ImageNet image samples, transfer learning assumptions still hold firmly. To understand what enables the ImageNet pretrained models to learn useful audio representations, we systematically study how much of pretrained weights is useful for learning spectrograms. We show (1) that for a given standard model using pretrained weights is better than using randomly initialized weights (2) qualitative results of what the CNNs learn from the spectrograms by visualizing the gradients. Besides, we show that even though we use the pretrained model weights for initialization, there is variance in performance in various output runs of the same model. This variance in performance is due to the random initialization of linear classification layer and random mini-batch orderings in multiple runs. This brings significant diversity to build stronger ensemble models with an overall improvement in accuracy. An ensemble of ImageNet pretrained DenseNet achieves 92.89% validation accuracy on the ESC-50 dataset and 87.42% validation accuracy on the UrbanSound8K dataset which is the current state-of-the-art on both of these datasets.