# TONet: Tone-Octave Network for Singing Melody Extraction from Polyphonic Music

arXiv: https://arxiv.org/abs/2202.00951 · HTML: https://mlsanity.com/paper/2202.00951/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 24 references that are in the map.
- First posted on arXiv 2 Feb 2022.

## Builds on (most cited first, 9 of 9)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Music SketchNet: Controllable Music Generation via Factorized Representations of Pitch and Rhythm](https://mlsanity.com/paper/2008.01291/) (2020)
- [Zero-shot Audio Source Separation through Query-based Learning from Weakly-labeled Data](https://mlsanity.com/paper/2112.07891/) (2021)
- [Continuous Melody Generation via Disentangled Short-Term Representations and Structural Conditions](https://mlsanity.com/paper/2002.02393/) (2020)
- [Vocal melody extraction using patch-based CNN](https://mlsanity.com/paper/1804.09202/) (2018)
- [A Streamlined Encoder/Decoder Architecture for Melody Extraction](https://mlsanity.com/paper/1810.12947/) (2018)
- [Learning Audio Embeddings with User Listening Data for Content-based Music Recommendation](https://mlsanity.com/paper/2010.15389/) (2020)

## Built on by (most cited first, 2 of 2)

- [Towards Improving Harmonic Sensitivity and Prediction Stability for Singing Melody Extraction](https://mlsanity.com/paper/2308.02723/) (2023)
- [A Mamba-based Network for Semi-supervised Singing Melody Extraction Using Confidence Binary Regularization](https://mlsanity.com/paper/2505.08681/) (2025)

## Abstract (arXiv)

Singing melody extraction is an important problem in the field of music information retrieval. Existing methods typically rely on frequency-domain representations to estimate the sung frequencies. However, this design does not lead to human-level performance in the perception of melody information for both tone (pitch-class) and octave. In this paper, we propose TONet, a plug-and-play model that improves both tone and octave perceptions by leveraging a novel input representation and a novel network architecture. First, we present an improved input representation, the Tone-CFP, that explicitly groups harmonics via a rearrangement of frequency-bins. Second, we introduce an encoder-decoder architecture that is designed to obtain a salience feature map, a tone feature map, and an octave feature map. Third, we propose a tone-octave fusion mechanism to improve the final salience feature map. Experiments are done to verify the capability of TONet with various baseline backbone models. Our results show that tone-octave fusion with Tone-CFP can significantly improve the singing voice extraction performance across various datasets -- with substantial gains in octave and tone accuracy.