# Extended playing techniques: The next milestone in musical instrument recognition

arXiv: https://arxiv.org/abs/1808.09730 · HTML: https://mlsanity.com/paper/1808.09730/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Aug 2018.

## Built on by (most cited first, 6 of 6)

- [Receptive Field Regularization Techniques for Audio Classification and Tagging with Deep Convolutional Neural Networks](https://mlsanity.com/paper/2105.12395/) (2021)
- [Leveraging Hierarchical Structures for Few-Shot Musical Instrument Recognition](https://mlsanity.com/paper/2107.07029/) (2021)
- [Playing Technique Detection by Fusing Note Onset Information in Guzheng Performance](https://mlsanity.com/paper/2209.08774/) (2022)
- [MERTech: Instrument Playing Technique Detection Using Self-Supervised Pretrained Model With Multi-Task Finetuning](https://mlsanity.com/paper/2310.09853/) (2023)
- [Differentiable Time-Frequency Scattering on GPU](https://mlsanity.com/paper/2204.08269/) (2022)
- [Timbre Classification of Musical Instruments with a Deep Learning Multi-Head Attention-Based Model](https://mlsanity.com/paper/2107.06231/) (2021)

## Abstract (arXiv)

The expressive variability in producing a musical note conveys information essential to the modeling of orchestration and style. As such, it plays a crucial role in computer-assisted browsing of massive digital music corpora. Yet, although the automatic recognition of a musical instrument from the recording of a single "ordinary" note is considered a solved problem, automatic identification of instrumental playing technique (IPT) remains largely underdeveloped. We benchmark machine listening systems for query-by-example browsing among 143 extended IPTs for 16 instruments, amounting to 469 triplets of instrument, mute, and technique. We identify and discuss three necessary conditions for significantly outperforming the traditional mel-frequency cepstral coefficient (MFCC) baseline: the addition of second-order scattering coefficients to account for amplitude modulation, the incorporation of long-range temporal dependencies, and metric learning using large-margin nearest neighbors (LMNN) to reduce intra-class variability. Evaluating on the Studio On Line (SOL) dataset, we obtain a precision at rank 5 of 99.7% for instrument recognition (baseline at 89.0%) and of 61.0% for IPT recognition (baseline at 44.5%). We interpret this gain through a qualitative assessment of practical usability and visualization using nonlinear dimensionality reduction.