# Timbre Analysis of Music Audio Signals with Convolutional Neural Networks

arXiv: https://arxiv.org/abs/1703.06697 · HTML: https://mlsanity.com/paper/1703.06697/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Mar 2017.

## Built on by (most cited first, 10 of 10)

- [Slow-Fast Auditory Streams For Audio Recognition](https://mlsanity.com/paper/2103.03516/) (2021)
- [NAAQA: A Neural Architecture for Acoustic Question Answering](https://mlsanity.com/paper/2106.06147/) (2021)
- [Deep Learning Based EDM Subgenre Classification using Mel-Spectrogram and Tempogram Features](https://mlsanity.com/paper/2110.08862/) (2021)
- [Myna: Masking-Based Contrastive Learning of Musical Representations](https://mlsanity.com/paper/2502.12511/) (2025)
- [Deep Convolutional and Recurrent Networks for Polyphonic Instrument Classification from Monophonic Raw Audio Waveforms](https://mlsanity.com/paper/2102.06930/) (2021)
- [Neuromorphic Auditory Perception by Neural Spiketrum](https://mlsanity.com/paper/2309.05430/) (2023)
- [Deep Insights into Cognitive Decline: A Survey of Leveraging Non-Intrusive Modalities with Deep Learning Techniques](https://mlsanity.com/paper/2410.18972/) (2024)
- [A Modulation Front-End for Music Audio Tagging](https://mlsanity.com/paper/2105.11836/) (2021)
- [Timbre Classification of Musical Instruments with a Deep Learning Multi-Head Attention-Based Model](https://mlsanity.com/paper/2107.06231/) (2021)
- [Pitch-Informed Instrument Assignment Using a Deep Convolutional Network with Multiple Kernel Shapes](https://mlsanity.com/paper/2107.13617/) (2021)

## Abstract (arXiv)

The focus of this work is to study how to efficiently tailor Convolutional Neural Networks (CNNs) towards learning timbre representations from log-mel magnitude spectrograms. We first review the trends when designing CNN architectures. Through this literature overview we discuss which are the crucial points to consider for efficiently learning timbre representations using CNNs. From this discussion we propose a design strategy meant to capture the relevant time-frequency contexts for learning timbre, which permits using domain knowledge for designing architectures. In addition, one of our main goals is to design efficient CNN architectures -- what reduces the risk of these models to over-fit, since CNNs' number of parameters is minimized. Several architectures based on the design principles we propose are successfully assessed for different research tasks related to timbre: singing voice phoneme classification, musical instrument recognition and music auto-tagging.