# Environmental Sound Deepfake Detection Using Deep-Learning Framework

arXiv: https://arxiv.org/abs/2604.19652 · HTML: https://mlsanity.com/paper/2604.19652/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 28 references that are in the map.
- First posted on arXiv 21 Apr 2026.

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [VGGSound: A Large-scale Audio-Visual Dataset](https://mlsanity.com/paper/2004.14368/) (2020)
- [Unsupervised Cross-lingual Representation Learning for Speech Recognition](https://mlsanity.com/paper/2006.13979/) (2020)
- [MLS: A Large-Scale Multilingual Dataset for Speech Research](https://mlsanity.com/paper/2012.03411/) (2020)
- [BEATs: Audio Pre-Training with Acoustic Tokenizers](https://mlsanity.com/paper/2212.09058/) (2022)
- [Clotho: An Audio Captioning Dataset](https://mlsanity.com/paper/1910.09387/) (2019)
- [Learning from Between-class Examples for Deep Sound Recognition](https://mlsanity.com/paper/1711.10282/) (2017)
- [Mixup-Based Acoustic Scene Classification Using Multi-Channel Convolutional Neural Network](https://mlsanity.com/paper/1805.07319/) (2018)
- [EnvSDD: Benchmarking Environmental Sound Deepfake Detection](https://mlsanity.com/paper/2505.19203/) (2025)
- [Multi-view Audio and Music Classification](https://mlsanity.com/paper/2103.02420/) (2021)
- [Wider or Deeper Neural Network Architecture for Acoustic Scene Classification with Mismatched Recording Devices](https://mlsanity.com/paper/2203.12314/) (2022)
- [SceneFake: An Initial Dataset and Benchmarks for Scene Fake Audio Detection](https://mlsanity.com/paper/2211.06073/) (2022)
- [SINCERE: Supervised Information Noise-Contrastive Estimation REvisited](https://mlsanity.com/paper/2309.14277/) (2023)
- [Robust Deep Learning Frameworks for Acoustic Scene and Respiratory Sound Classification](https://mlsanity.com/paper/2107.09268/) (2021)
- [Detection of Deepfake Environmental Audio](https://mlsanity.com/paper/2403.17529/) (2024)
- [DIN-CTS: Low-Complexity Depthwise-Inception Neural Network with Contrastive Training Strategy for Deepfake Speech Detection](https://mlsanity.com/paper/2502.20225/) (2025)
- [Environmental Sound Deepfake Detection Challenge: An Overview](https://mlsanity.com/paper/2512.24140/) (2025)
- [Representation Loss Minimization with Randomized Selection Strategy for Efficient Environmental Fake Audio Detection](https://mlsanity.com/paper/2409.15767/) (2024)

## Abstract (arXiv)

In this paper, we propose a deep-learning framework for Environmental Sound Deepfake Detection (ESDD) - the task of identifying whether the sound scene and sound event in an input audio recording is fake or real. To this end, we first conduct extensive experiments to explore how individual spectrograms, a wide range of network architectures, and pre-trained models affect the performance of an ESDD model. The experimental results on the benchmark datasets of EnvSDD indicate that detecting deepfake audio of sound scenes and detecting deepfake audio of sound events should be considered as individual tasks. We also show that fine-tuning a pre-trained model is more effective than training a model from scratch for ESDD. Ultimately, our best model, which fine-tunes the pre-trained BEATs model using the proposed two-phase training strategy, achieves an Accuracy of 0.98, F1 score of 0.95, and AUC score of 0.99 on the Test subset of the EnvSDD dataset. Our best model also achieves an Accuracy of 0.86, F1 score of 0.80, and AUC of 0.93 when evaluated cross-dataset on the ESD-Challenge-TestSet dataset.