# A Phoneme-Informed Neural Network Model for Note-Level Singing Transcription

arXiv: https://arxiv.org/abs/2304.05917 · HTML: https://mlsanity.com/paper/2304.05917/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 28 references that are in the map.
- First posted on arXiv 12 Apr 2023.

## Builds on (most cited first, 7 of 7)

- [Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Dataset](https://mlsanity.com/paper/1810.12247/) (2018)
- [Onsets and Frames: Dual-Objective Piano Transcription](https://mlsanity.com/paper/1710.11153/) (2017)
- [High-resolution Piano Transcription with Pedals by Regressing Onset and Offset Times](https://mlsanity.com/paper/2010.01815/) (2020)
- [On the Potential of Simple Framewise Approaches to Piano Transcription](https://mlsanity.com/paper/1612.05153/) (2016)
- [DALI: a large Dataset of synchronized Audio, LyrIcs and notes, automatically created using teacher-student machine learning paradigm](https://mlsanity.com/paper/1906.10606/) (2019)
- [Omnizart: A General Toolbox for Automatic Music Transcription](https://mlsanity.com/paper/2106.00497/) (2021)
- [Polyphonic Piano Transcription Using Autoregressive Multi-State Note Model](https://mlsanity.com/paper/2010.01104/) (2020)

## Built on by (most cited first, 5 of 5)

- [Toward Leveraging Pre-Trained Self-Supervised Frontends for Automatic Singing Voice Understanding Tasks: Three Case Studies](https://mlsanity.com/paper/2306.12714/) (2023)
- [Human Voice Pitch Estimation: A Convolutional Network with Auto-Labeled and Synthetic Data](https://mlsanity.com/paper/2308.07170/) (2023)
- [A Real-Time Lyrics Alignment System Using Chroma And Phonetic Features For Classical Vocal Performance](https://mlsanity.com/paper/2401.09200/) (2024)
- [Source Separation for A Cappella Music](https://mlsanity.com/paper/2509.26580/) (2025)
- [BERT-APC: A Reference-free Framework for Automatic Pitch Correction via Musical Context Inference](https://mlsanity.com/paper/2511.20006/) (2025)

## Abstract (arXiv)

Note-level automatic music transcription is one of the most representative music information retrieval (MIR) tasks and has been studied for various instruments to understand music. However, due to the lack of high-quality labeled data, transcription of many instruments is still a challenging task. In particular, in the case of singing, it is difficult to find accurate notes due to its expressiveness in pitch, timbre, and dynamics. In this paper, we propose a method of finding note onsets of singing voice more accurately by leveraging the linguistic characteristics of singing, which are not seen in other instruments. The proposed model uses mel-scaled spectrogram and phonetic posteriorgram (PPG), a frame-wise likelihood of phoneme, as an input of the onset detection network while PPG is generated by the pre-trained network with singing and speech data. To verify how linguistic features affect onset detection, we compare the evaluation results through the dataset with different languages and divide onset types for detailed analysis. Our approach substantially improves the performance of singing transcription and therefore emphasizes the importance of linguistic features in singing analysis.