# An Attention Mechanism for Musical Instrument Recognition

arXiv: https://arxiv.org/abs/1907.04294 · HTML: https://mlsanity.com/paper/1907.04294/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jul 2019.

## Built on by (most cited first, 14 of 14)

- [Receptive Field Regularization Techniques for Audio Classification and Tagging with Deep Convolutional Neural Networks](https://mlsanity.com/paper/2105.12395/) (2021)
- [Leveraging Hierarchical Structures for Few-Shot Musical Instrument Recognition](https://mlsanity.com/paper/2107.07029/) (2021)
- [OpenMU: Your Swiss Army Knife for Music Understanding](https://mlsanity.com/paper/2410.15573/) (2024)
- [Music Foundation Model as Generic Booster for Music Downstream Tasks](https://mlsanity.com/paper/2411.01135/) (2024)
- [Audio Embeddings as Teachers for Music Classification](https://mlsanity.com/paper/2306.17424/) (2023)
- [Music Instrument Classification Reprogrammed](https://mlsanity.com/paper/2211.08379/) (2022)
- [Deep Convolutional and Recurrent Networks for Polyphonic Instrument Classification from Monophonic Raw Audio Waveforms](https://mlsanity.com/paper/2102.06930/) (2021)
- [MIRFLEX: Music Information Retrieval Feature Library for Extraction](https://mlsanity.com/paper/2411.00469/) (2024)
- [DeepResonance: Enhancing Multimodal Music Understanding via Music-centric Multi-way Instruction Tuning](https://mlsanity.com/paper/2502.12623/) (2025)
- [Self-refining of Pseudo Labels for Music Source Separation with Noisy Labeled Data](https://mlsanity.com/paper/2307.12576/) (2023)
- [On the choice of the optimal temporal support for audio classification with Pre-trained embeddings](https://mlsanity.com/paper/2312.14005/) (2023)
- [Timbre Classification of Musical Instruments with a Deep Learning Multi-Head Attention-Based Model](https://mlsanity.com/paper/2107.06231/) (2021)
- [Pitch-Informed Instrument Assignment Using a Deep Convolutional Network with Multiple Kernel Shapes](https://mlsanity.com/paper/2107.13617/) (2021)
- [Instrument Classification of Solo Sheet Music Images](https://mlsanity.com/paper/2609.18980/) (2026)

## Abstract (arXiv)

While the automatic recognition of musical instruments has seen significant progress, the task is still considered hard for music featuring multiple instruments as opposed to single instrument recordings. Datasets for polyphonic instrument recognition can be categorized into roughly two categories. Some, such as MedleyDB, have strong per-frame instrument activity annotations but are usually small in size. Other, larger datasets such as OpenMIC only have weak labels, i.e., instrument presence or absence is annotated only for long snippets of a song. We explore an attention mechanism for handling weakly labeled data for multi-label instrument recognition. Attention has been found to perform well for other tasks with weakly labeled data. We compare the proposed attention model to multiple models which include a baseline binary relevance random forest, recurrent neural network, and fully connected neural networks. Our results show that incorporating attention leads to an overall improvement in classification accuracy metrics across all 20 instruments in the OpenMIC dataset. We find that attention enables models to focus on (or `attend to') specific time segments in the audio relevant to each instrument label leading to interpretable results.