# ESResNet: Environmental Sound Classification Based on Visual Domain Models

arXiv: https://arxiv.org/abs/2004.07301 · HTML: https://mlsanity.com/paper/2004.07301/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Apr 2020.

## Built on by (most cited first, 30 of 31)

- [AST: Audio Spectrogram Transformer](https://mlsanity.com/paper/2104.01778/) (2021)
- [AudioCLIP: Extending CLIP to Image, Text and Audio](https://mlsanity.com/paper/2106.13043/) (2021)
- [PSLA: Improving Audio Tagging with Pretraining, Sampling, Labeling, and Aggregation](https://mlsanity.com/paper/2102.01243/) (2021)
- [Audiovisual Masked Autoencoders](https://mlsanity.com/paper/2212.05922/) (2022)
- [Emotional Speech-driven 3D Body Animation via Disentangled Latent Diffusion](https://mlsanity.com/paper/2312.04466/) (2023)
- [Accommodating Audio Modality in CLIP for Multimodal Processing](https://mlsanity.com/paper/2303.06591/) (2023)
- [Connecting the Dots between Audio and Text without Parallel Data through Visual Knowledge Transfer](https://mlsanity.com/paper/2112.08995/) (2021)
- [Interactive Audio-text Representation for Automated Audio Captioning with Contrastive Learning](https://mlsanity.com/paper/2203.15526/) (2022)
- [CMKD: CNN/Transformer-Based Cross-Model Knowledge Distillation for Audio Classification](https://mlsanity.com/paper/2203.06760/) (2022)
- [Multi-view Audio and Music Classification](https://mlsanity.com/paper/2103.02420/) (2021)
- [SampleMatch: Drum Sample Retrieval by Musical Context](https://mlsanity.com/paper/2208.01141/) (2022)
- [Detection of Audio-Video Synchronization Errors Via Event Detection](https://mlsanity.com/paper/2104.10116/) (2021)
- [Environmental Sound Classification on the Edge: A Pipeline for Deep Acoustic Networks on Extremely Resource-Constrained Devices](https://mlsanity.com/paper/2103.03483/) (2021)
- [SoundCLR: Contrastive Learning of Representations For Improved Environmental Sound Classification](https://mlsanity.com/paper/2103.01929/) (2021)
- [Effective Audio Classification Network Based on Paired Inverse Pyramid Structure and Dense MLP Block](https://mlsanity.com/paper/2211.02940/) (2022)
- [AudRandAug: Random Image Augmentations for Audio Classification](https://mlsanity.com/paper/2309.04762/) (2023)
- [Comparison of semi-supervised deep learning algorithms for audio classification](https://mlsanity.com/paper/2102.08183/) (2021)
- [Bridging High-Quality Audio and Video via Language for Sound Effects Retrieval from Visual Queries](https://mlsanity.com/paper/2308.09089/) (2023)
- [OneEncoder: A Lightweight Framework for Progressive Alignment of Modalities](https://mlsanity.com/paper/2409.11059/) (2024)
- [NeuroView: Explainable Deep Network Decision Making](https://mlsanity.com/paper/2110.07778/) (2021)
- [Sound Tagging in Infant-centric Home Soundscapes](https://mlsanity.com/paper/2406.17190/) (2024)
- [Unsupervised Discriminative Learning of Sounds for Audio Event Classification](https://mlsanity.com/paper/2105.09279/) (2021)
- [Maximizing Audio Event Detection Model Performance on Small Datasets Through Knowledge Transfer, Data Augmentation, And Pretraining: An Ablation Study](https://mlsanity.com/paper/2202.03514/) (2022)
- [End-to-End Audio Strikes Back: Boosting Augmentations Towards An Efficient Audio Classification Network](https://mlsanity.com/paper/2204.11479/) (2022)
- [GAFX: A General Audio Feature eXtractor](https://mlsanity.com/paper/2207.09145/) (2022)
- [Formal Verification of Long Short-Term Memory based Audio Classifiers: A Star based Approach](https://mlsanity.com/paper/2311.12130/) (2023)
- [Spectro-ViT: A Vision Transformer Model for GABA-edited MRS Reconstruction Using Spectrograms](https://mlsanity.com/paper/2311.15386/) (2023)
- [Transfer Learning in Vocal Education: Technical Evaluation of Limited Samples Describing Mezzo-soprano](https://mlsanity.com/paper/2410.23325/) (2024)
- [How Class Ontology and Data Scale Affect Audio Transfer Learning](https://mlsanity.com/paper/2603.25476/) (2026)
- [DAStatFormer: A Hybrid Multibranch Transformer with Statistical Feature Integration for DAS-Based Pattern Recognitions](https://mlsanity.com/paper/2606.00081/) (2026)

## Abstract (arXiv)

Environmental Sound Classification (ESC) is an active research area in the audio domain and has seen a lot of progress in the past years. However, many of the existing approaches achieve high accuracy by relying on domain-specific features and architectures, making it harder to benefit from advances in other fields (e.g., the image domain). Additionally, some of the past successes have been attributed to a discrepancy of how results are evaluated (i.e., on unofficial splits of the UrbanSound8K (US8K) dataset), distorting the overall progression of the field. The contribution of this paper is twofold. First, we present a model that is inherently compatible with mono and stereo sound inputs. Our model is based on simple log-power Short-Time Fourier Transform (STFT) spectrograms and combines them with several well-known approaches from the image domain (i.e., ResNet, Siamese-like networks and attention). We investigate the influence of cross-domain pre-training, architectural changes, and evaluate our model on standard datasets. We find that our model out-performs all previously known approaches in a fair comparison by achieving accuracies of 97.0 % (ESC-10), 91.5 % (ESC-50) and 84.2 % / 85.4 % (US8K mono / stereo). Second, we provide a comprehensive overview of the actual state of the field, by differentiating several previously reported results on the US8K dataset between official or unofficial splits. For better reproducibility, our code (including any re-implementations) is made available.