# XAI-based Comparison of Input Representations for Audio Event Classification

arXiv: https://arxiv.org/abs/2304.14019 · HTML: https://mlsanity.com/paper/2304.14019/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 30 references that are in the map.
- First posted on arXiv 27 Apr 2023.

## Builds on (most cited first, 11 of 11)

- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [CNN Architectures for Large-Scale Audio Classification](https://mlsanity.com/paper/1609.09430/) (2016)
- [Explaining NonLinear Classification Decisions with Deep Taylor Decomposition](https://mlsanity.com/paper/1512.02479/) (2015)
- [Towards Best Practice in Explaining Neural Network Decisions with LRP](https://mlsanity.com/paper/1910.09840/) (2019)
- [Software for Dataset-wide XAI: From Local Explanations to Global Insights with Zennit, CoRelAy, and ViRelAy](https://mlsanity.com/paper/2106.13200/) (2021)
- [Multi-Format Contrastive Learning of Audio Representations](https://mlsanity.com/paper/2103.06508/) (2021)
- [Explainable AI for Time Series via Virtual Inspection Layers](https://mlsanity.com/paper/2303.06365/) (2023)
- [End-to-End Environmental Sound Classification using a 1D Convolutional Neural Network](https://mlsanity.com/paper/1904.08990/) (2019)
- [Raw Waveform-based Audio Classification Using Sample-level CNN Architectures](https://mlsanity.com/paper/1712.00866/) (2017)
- [Emergency Vehicles Audio Detection and Localization in Autonomous Driving](https://mlsanity.com/paper/2109.14797/) (2021)
- [Interpreting deep urban sound classification using Layer-wise Relevance Propagation](https://mlsanity.com/paper/2111.10235/) (2021)

## Built on by (most cited first, 3 of 3)

- [Explaining Speech Classification Models via Word-Level Audio Segments and Paralinguistic Features](https://mlsanity.com/paper/2309.07733/) (2023)
- [Audio Explanation Synthesis with Generative Foundation Models](https://mlsanity.com/paper/2410.07530/) (2024)
- [AudioGenX: Explainability on Text-to-Audio Generative Models](https://mlsanity.com/paper/2502.00459/) (2025)

## Abstract (arXiv)

Deep neural networks are a promising tool for Audio Event Classification. In contrast to other data like natural images, there are many sensible and non-obvious representations for audio data, which could serve as input to these models. Due to their black-box nature, the effect of different input representations has so far mostly been investigated by measuring classification performance. In this work, we leverage eXplainable AI (XAI), to understand the underlying classification strategies of models trained on different input representations. Specifically, we compare two model architectures with regard to relevant input features used for Audio Event Detection: one directly processes the signal as the raw waveform, and the other takes in its time-frequency spectrogram representation. We show how relevance heatmaps obtained via "Siren"{Layer-wise Relevance Propagation} uncover representation-dependent decision strategies. With these insights, we can make a well-informed decision about the best input representation in terms of robustness and representativity and confirm that the model's classification strategies align with human requirements.