# Timbre Classification of Musical Instruments with a Deep Learning Multi-Head Attention-Based Model

arXiv: https://arxiv.org/abs/2107.06231 · HTML: https://mlsanity.com/paper/2107.06231/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 35 references that are in the map.
- First posted on arXiv 13 Jul 2021.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Are Sixteen Heads Really Better than One?](https://mlsanity.com/paper/1905.10650/) (2019)
- [Learning Features of Music from Scratch](https://mlsanity.com/paper/1611.09827/) (2016)
- [Automatic tagging using deep convolutional neural networks](https://mlsanity.com/paper/1606.00298/) (2016)
- [Learning Disentangled Representations of Timbre and Pitch for Musical Instrument Sounds Using Gaussian Mixture Variational Autoencoders](https://mlsanity.com/paper/1906.08152/) (2019)
- [An Attention Mechanism for Musical Instrument Recognition](https://mlsanity.com/paper/1907.04294/) (2019)
- [Deep convolutional neural networks for predominant instrument recognition in polyphonic music](https://mlsanity.com/paper/1605.09507/) (2016)
- [Timbre Analysis of Music Audio Signals with Convolutional Neural Networks](https://mlsanity.com/paper/1703.06697/) (2017)
- [Frame-level Instrument Recognition by Timbre and Pitch](https://mlsanity.com/paper/1806.09587/) (2018)
- [Extended playing techniques: The next milestone in musical instrument recognition](https://mlsanity.com/paper/1808.09730/) (2018)
- [Augmentation Methods on Monophonic Audio for Instrument Classification in Polyphonic Music](https://mlsanity.com/paper/1911.12505/) (2019)
- [Automatic Instrument Recognition in Polyphonic Music Using Convolutional Neural Networks](https://mlsanity.com/paper/1511.05520/) (2015)

## Abstract (arXiv)

The aim of this work is to define a model based on deep learning that is able to identify different instrument timbres with as few parameters as possible. For this purpose, we have worked with classical orchestral instruments played with different dynamics, which are part of a few instrument families and which play notes in the same pitch range. It has been possible to assess the ability to classify instruments by timbre even if the instruments are playing the same note with the same intensity. The network employed uses a multi-head attention mechanism, with 8 heads and a dense network at the output taking as input the log-mel magnitude spectrograms of the sound samples. This network allows the identification of 20 instrument classes of the classical orchestra, achieving an overall F1 value of 0.62. An analysis of the weights of the attention layer has been performed and the confusion matrix of the model is presented, allowing us to assess the ability of the proposed architecture to distinguish timbre and to establish the aspects on which future work should focus.