# Towards Improving Harmonic Sensitivity and Prediction Stability for Singing Melody Extraction

arXiv: https://arxiv.org/abs/2308.02723 · HTML: https://mlsanity.com/paper/2308.02723/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 24 references that are in the map.
- First posted on arXiv 4 Aug 2023.

## Builds on (most cited first, 8 of 8)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [CREPE: A Convolutional Representation for Pitch Estimation](https://mlsanity.com/paper/1802.06182/) (2018)
- [High-resolution Piano Transcription with Pedals by Regressing Onset and Offset Times](https://mlsanity.com/paper/2010.01815/) (2020)
- [SpecTNT: a Time-Frequency Transformer for Music Audio](https://mlsanity.com/paper/2110.09127/) (2021)
- [Vocal melody extraction using patch-based CNN](https://mlsanity.com/paper/1804.09202/) (2018)
- [A Streamlined Encoder/Decoder Architecture for Melody Extraction](https://mlsanity.com/paper/1810.12947/) (2018)
- [Learning Audio Embeddings with User Listening Data for Content-based Music Recommendation](https://mlsanity.com/paper/2010.15389/) (2020)
- [TONet: Tone-Octave Network for Singing Melody Extraction from Polyphonic Music](https://mlsanity.com/paper/2202.00951/) (2022)

## Built on by (most cited first, 2 of 2)

- [kNN-SVC: Robust Zero-Shot Singing Voice Conversion with Additive Synthesis and Concatenation Smoothness Optimization](https://mlsanity.com/paper/2504.05686/) (2025)
- [EGGCodec: A Robust Neural Encodec Framework for EGG Reconstruction and F0 Extraction](https://mlsanity.com/paper/2508.08924/) (2025)

## Abstract (arXiv)

In deep learning research, many melody extraction models rely on redesigning neural network architectures to improve performance. In this paper, we propose an input feature modification and a training objective modification based on two assumptions. First, harmonics in the spectrograms of audio data decay rapidly along the frequency axis. To enhance the model's sensitivity on the trailing harmonics, we modify the Combined Frequency and Periodicity (CFP) representation using discrete z-transform. Second, the vocal and non-vocal segments with extremely short duration are uncommon. To ensure a more stable melody contour, we design a differentiable loss function that prevents the model from predicting such segments. We apply these modifications to several models, including MSNet, FTANet, and a newly introduced model, PianoNet, modified from a piano transcription network. Our experimental results demonstrate that the proposed modifications are empirically effective for singing melody extraction.