# Deep convolutional neural networks for predominant instrument recognition in polyphonic music

arXiv: https://arxiv.org/abs/1605.09507 · HTML: https://mlsanity.com/paper/1605.09507/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 May 2016.

## Built on by (most cited first, 11 of 11)

- [Leveraging Hierarchical Structures for Few-Shot Musical Instrument Recognition](https://mlsanity.com/paper/2107.07029/) (2021)
- [VPN: Video Provenance Network for Robust Content Attribution](https://mlsanity.com/paper/2109.10038/) (2021)
- [Show Me the Instruments: Musical Instrument Retrieval from Mixture Audio](https://mlsanity.com/paper/2211.07951/) (2022)
- [A Comparative Study of Speaker Role Identification in Air Traffic Communication Using Deep Learning Approaches](https://mlsanity.com/paper/2111.02041/) (2021)
- [Deep Convolutional and Recurrent Networks for Polyphonic Instrument Classification from Monophonic Raw Audio Waveforms](https://mlsanity.com/paper/2102.06930/) (2021)
- [Self-refining of Pseudo Labels for Music Source Separation with Noisy Labeled Data](https://mlsanity.com/paper/2307.12576/) (2023)
- [Timbre Classification of Musical Instruments with a Deep Learning Multi-Head Attention-Based Model](https://mlsanity.com/paper/2107.06231/) (2021)
- [Pitch-Informed Instrument Assignment Using a Deep Convolutional Network with Multiple Kernel Shapes](https://mlsanity.com/paper/2107.13617/) (2021)
- [AI and Tempo Estimation: A Review](https://mlsanity.com/paper/2401.00209/) (2023)
- [Music Genre Classification using Large Language Models](https://mlsanity.com/paper/2410.08321/) (2024)
- [Instrument Classification of Solo Sheet Music Images](https://mlsanity.com/paper/2609.18980/) (2026)

## Abstract (arXiv)

Identifying musical instruments in polyphonic music recordings is a challenging but important problem in the field of music information retrieval. It enables music search by instrument, helps recognize musical genres, or can make music transcription easier and more accurate. In this paper, we present a convolutional neural network framework for predominant instrument recognition in real-world polyphonic music. We train our network from fixed-length music excerpts with a single-labeled predominant instrument and estimate an arbitrary number of predominant instruments from an audio signal with a variable length. To obtain the audio-excerpt-wise result, we aggregate multiple outputs from sliding windows over the test audio. In doing so, we investigated two different aggregation methods: one takes the average for each instrument and the other takes the instrument-wise sum followed by normalization. In addition, we conducted extensive experiments on several important factors that affect the performance, including analysis window size, identification threshold, and activation functions for neural networks to find the optimal set of parameters. Using a dataset of 10k audio excerpts from 11 instruments for evaluation, we found that convolutional neural networks are more robust than conventional methods that exploit spectral features and source separation with support vector machines. Experimental results showed that the proposed convolutional network architecture obtained an F1 measure of 0.602 for micro and 0.503 for macro, respectively, achieving 19.6% and 16.4% in performance improvement compared with other state-of-the-art algorithms.