# Pitch-Informed Instrument Assignment Using a Deep Convolutional Network with Multiple Kernel Shapes

arXiv: https://arxiv.org/abs/2107.13617 · HTML: https://mlsanity.com/paper/2107.13617/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 24 references that are in the map.
- First posted on arXiv 28 Jul 2021.

## Builds on (most cited first, 11 of 11)

- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Learning Features of Music from Scratch](https://mlsanity.com/paper/1611.09827/) (2016)
- [Simultaneous Separation and Transcription of Mixtures with Multiple Polyphonic and Percussive Instruments](https://mlsanity.com/paper/1910.12621/) (2019)
- [Multi-scale Multi-band DenseNets for Audio Source Separation](https://mlsanity.com/paper/1706.09588/) (2017)
- [An Attention Mechanism for Musical Instrument Recognition](https://mlsanity.com/paper/1907.04294/) (2019)
- [Deep convolutional neural networks for predominant instrument recognition in polyphonic music](https://mlsanity.com/paper/1605.09507/) (2016)
- [Timbre Analysis of Music Audio Signals with Convolutional Neural Networks](https://mlsanity.com/paper/1703.06697/) (2017)
- [Multitask learning for frame-level instrument recognition](https://mlsanity.com/paper/1811.01143/) (2018)
- [Frame-level Instrument Recognition by Timbre and Pitch](https://mlsanity.com/paper/1806.09587/) (2018)
- [Investigating kernel shapes and skip connections for deep learning-based harmonic-percussive separation](https://mlsanity.com/paper/1905.01899/) (2019)
- [Multitask Learning for Fundamental Frequency Estimation in Music](https://mlsanity.com/paper/1809.00381/) (2018)

## Abstract (arXiv)

This paper proposes a deep convolutional neural network for performing note-level instrument assignment. Given a polyphonic multi-instrumental music signal along with its ground truth or predicted notes, the objective is to assign an instrumental source for each note. This problem is addressed as a pitch-informed classification task where each note is analysed individually. We also propose to utilise several kernel shapes in the convolutional layers in order to facilitate learning of efficient timbre-discriminative feature maps. Experiments on the MusicNet dataset using 7 instrument classes show that our approach is able to achieve an average F-score of 0.904 when the original multi-pitch annotations are used as the pitch information for the system, and that it also excels if the note information is provided using third-party multi-pitch estimation algorithms. We also include ablation studies investigating the effects of the use of multiple kernel shapes and comparing different input representations for the audio and the note-related information.