# Perceptual Musical Features for Interpretable Audio Tagging

arXiv: https://arxiv.org/abs/2312.11234 · HTML: https://mlsanity.com/paper/2312.11234/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 41 references that are in the map.
- First posted on arXiv 18 Dec 2023.

## Builds on (most cited first, 15 of 15)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [Codified audio language modeling learns useful representations for music information retrieval](https://mlsanity.com/paper/2107.05677/) (2021)
- [Ablation Studies in Artificial Neural Networks](https://mlsanity.com/paper/1901.08644/) (2019)
- [Evaluation of CNN-based Automatic Music Tagging Models](https://mlsanity.com/paper/2006.00751/) (2020)
- [Toward Interpretable Music Tagging with Self-Attention](https://mlsanity.com/paper/1906.04972/) (2019)
- [End-to-end learning for music audio tagging at scale](https://mlsanity.com/paper/1711.02520/) (2017)
- [Masked Modeling Duo: Learning Representations by Encouraging Both Networks to Model the Input](https://mlsanity.com/paper/2210.14648/) (2022)
- [Omnizart: A General Toolbox for Automatic Music Transcription](https://mlsanity.com/paper/2106.00497/) (2021)
- [Towards Explainable Music Emotion Recognition: The Route via Mid-level Features](https://mlsanity.com/paper/1907.03572/) (2019)
- [A data-driven approach to mid-level perceptual musical feature modeling](https://mlsanity.com/paper/1806.04903/) (2018)
- [Tracing Back Music Emotion Predictions to Sound Sources and Intuitive Perceptual Qualities](https://mlsanity.com/paper/2106.07787/) (2021)
- [Towards Explaining Expressive Qualities in Piano Recordings: Transfer of Explanatory Features via Acoustic Domain Adaptation](https://mlsanity.com/paper/2102.13479/) (2021)
- [Employing Crowdsourcing for Enriching a Music Knowledge Base in Higher Education](https://mlsanity.com/paper/2306.07310/) (2023)

## Built on by (most cited first, 7 of 7)

- [Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2412.02104/) (2024)
- [MusicLIME: Explainable Multimodal Music Understanding](https://mlsanity.com/paper/2409.10496/) (2024)
- [Music Interpretation and Emotion Perception: A Computational and Neurophysiological Investigation](https://mlsanity.com/paper/2506.01982/) (2025)
- [CHORDONOMICON: A Dataset of 666,000 Songs and their Chord Progressions](https://mlsanity.com/paper/2410.22046/) (2024)
- [Semantic-Aware Interpretable Multimodal Music Auto-Tagging](https://mlsanity.com/paper/2505.17233/) (2025)
- [Exploring How Audio Effects Alter Emotion with Foundation Models](https://mlsanity.com/paper/2509.15151/) (2025)
- [Constructing Composite Features for Interpretable Music-Tagging](https://mlsanity.com/paper/2603.28644/) (2026)

## Abstract (arXiv)

In the age of music streaming platforms, the task of automatically tagging music audio has garnered significant attention, driving researchers to devise methods aimed at enhancing performance metrics on standard datasets. Most recent approaches rely on deep neural networks, which, despite their impressive performance, possess opacity, making it challenging to elucidate their output for a given input. While the issue of interpretability has been emphasized in other fields like medicine, it has not received attention in music-related tasks. In this study, we explored the relevance of interpretability in the context of automatic music tagging. We constructed a workflow that incorporates three different information extraction techniques: a) leveraging symbolic knowledge, b) utilizing auxiliary deep neural networks, and c) employing signal processing to extract perceptual features from audio files. These features were subsequently used to train an interpretable machine-learning model for tag prediction. We conducted experiments on two datasets, namely the MTG-Jamendo dataset and the GTZAN dataset. Our method surpassed the performance of baseline models in both tasks and, in certain instances, demonstrated competitiveness with the current state-of-the-art. We conclude that there are use cases where the deterioration in performance is outweighed by the value of interpretability.