# Omnizart: A General Toolbox for Automatic Music Transcription

arXiv: https://arxiv.org/abs/2106.00497 · HTML: https://mlsanity.com/paper/2106.00497/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 25 references that are in the map.
- First posted on arXiv 1 Jun 2021.

## Builds on (most cited first, 10 of 10)

- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [Image Transformer](https://mlsanity.com/paper/1802.05751/) (2018)
- [Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Dataset](https://mlsanity.com/paper/1810.12247/) (2018)
- [madmom: a new Python Audio and Music Signal Processing Library](https://mlsanity.com/paper/1605.07008/) (2016)
- [Learning Features of Music from Scratch](https://mlsanity.com/paper/1611.09827/) (2016)
- [ShakeDrop Regularization for Deep Residual Learning](https://mlsanity.com/paper/1802.02375/) (2018)
- [On the Potential of Simple Framewise Approaches to Piano Transcription](https://mlsanity.com/paper/1612.05153/) (2016)
- [Invariances and Data Augmentation for Supervised Music Transcription](https://mlsanity.com/paper/1711.04845/) (2017)
- [Vocal melody extraction using patch-based CNN](https://mlsanity.com/paper/1804.09202/) (2018)

## Built on by (most cited first, 13 of 13)

- [Video2Music: Suitable Music Generation from Videos using an Affective Multimodal Transformer model](https://mlsanity.com/paper/2311.00968/) (2023)
- [Content-based Controls For Music Large Language Modeling](https://mlsanity.com/paper/2310.17162/) (2023)
- [High Resolution Guitar Transcription via Domain Adaptation](https://mlsanity.com/paper/2402.15258/) (2024)
- [Perceptual Musical Features for Interpretable Audio Tagging](https://mlsanity.com/paper/2312.11234/) (2023)
- [Jointist: Joint Learning for Multi-instrument Transcription and Its Applications](https://mlsanity.com/paper/2206.10805/) (2022)
- [A Phoneme-Informed Neural Network Model for Note-Level Singing Transcription](https://mlsanity.com/paper/2304.05917/) (2023)
- [Jointist: Simultaneous Improvement of Multi-instrument Transcription and Music Source Separation via Joint Training](https://mlsanity.com/paper/2302.00286/) (2023)
- [MOSA: Music Motion with Semantic Annotation Dataset for Cross-Modal Music Processing](https://mlsanity.com/paper/2406.06375/) (2024)
- [STRUM: A Spectral Transcription and Rhythm Understanding Model for End-to-End Generation of Playable Rhythm-Game Charts](https://mlsanity.com/paper/2605.12135/) (2026)
- [Semantic-Aware Interpretable Multimodal Music Auto-Tagging](https://mlsanity.com/paper/2505.17233/) (2025)
- [Constructing Composite Features for Interpretable Music-Tagging](https://mlsanity.com/paper/2603.28644/) (2026)
- [FIGMA: Towards FIne-Grained Music retrievAl](https://mlsanity.com/paper/2606.06615/) (2026)
- [Generalized Audio-Driven Synthesis of Precise Drummer Motion](https://mlsanity.com/paper/2608.19055/) (2026)

## Abstract (arXiv)

We present and release Omnizart, a new Python library that provides a streamlined solution to automatic music transcription (AMT). Omnizart encompasses modules that construct the life-cycle of deep learning-based AMT, and is designed for ease of use with a compact command-line interface. To the best of our knowledge, Omnizart is the first transcription toolkit which offers models covering a wide class of instruments ranging from solo, instrument ensembles, percussion instruments to vocal, as well as models for chord recognition and beat/downbeat tracking, two music information retrieval (MIR) tasks highly related to AMT.