YingMusic-Singer: Zero-shot Singing Voice Synthesis and Editing with Annotation-free Melody Guidance
Not on the map. Why?
Abstract (arXiv)
Singing Voice Synthesis (SVS) remains constrained in practical deployment due to its strong dependence on accurate phoneme-level alignment and manually annotated melody contours, requirements that are resource-intensive and hinder scalability. To overcome these limitations, we propose a melody-driven SVS framework capable of synthesizing arbitrary lyrics following any reference melody, without relying on phoneme-level alignment. Our method builds on a Diffusion Transformer (DiT) architecture, enhanced with a dedicated melody extraction module that derives melody representations directly from reference audio. To ensure robust melody encoding, we employ a teacher model to guide the optimization of the melody extractor, alongside an implicit alignment mechanism that enforces similarity distribution constraints for improved melodic stability and coherence. Additionally, we refine duration modeling using weakly annotated song data and introduce a Flow-GRPO reinforcement learning strategy with a multi-objective reward function to jointly enhance pronunciation clarity and melodic fidelity. Experiments show that our model achieves superior performance over existing approaches in both objective measures and subjective listening tests, especially in zero-shot and lyric adaptation settings, while maintaining high audio quality without manual annotation. This work offers a practical and scalable solution for advancing data-efficient singing voice synthesis. To support reproducibility, we release our inference code and model checkpoints.
Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 38 of its 43 references that are in the map. First posted on arXiv 4 Dec 2025.
Builds on
38 of 43 references.
References · 38
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
- 2020Learning to summarize from human feedbackcited by 2,562
- 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
- 2025Flow-GRPO: Training Flow Matching Models via Online RLcited by 605
- 2025DanceGRPO: Unleashing GRPO on Visual Generationcited by 388
- 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matchingcited by 353
- 2020DNSMOS: A Non-Intrusive Perceptual Objective Speech Quality metric to evaluate Noise Suppressorscited by 235
- 2021DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanismcited by 154
- 2022Reliability of CKA as a Similarity Measure in Deep Learningcited by 82
26 more
- 2025LongCat-Video Technical Reportcited by 73
- 2021Multi-Singer: Fast Multi-Singer Singing Voice Vocoder With A Large-Scale Corpuscited by 60
- 2022Opencpop: A High-Quality Open Source Chinese Popular Song Corpus for Singing Voice Synthesiscited by 53
- 2025DiffRhythm: Blazingly Fast and Embarrassingly Simple End-to-End Full-Length Song Generation with Latent Diffusioncited by 46
- 2025FireRedASR: Open-Source Industrial-Grade Mandarin Speech Recognition Models from Encoder-Decoder to LLM Integrationcited by 44
- 2020HiFiSinger: Towards High-Fidelity Neural Singing Voice Synthesiscited by 39
- 2021VISinger: Variational Inference with Adversarial Learning for End-to-End Singing Voice Synthesiscited by 38
- 2025Vevo: Controllable Zero-Shot Voice Imitation with Self-Supervised Disentanglementcited by 34
- 2020XiaoiceSing: A High-Quality and Integrated Singing Voice Synthesis Systemcited by 33
- 2024GTSinger: A Global Multi-Technique Singing Corpus with Realistic Music Scores for All Singing Taskscited by 29
- 2023RMVPE: A Robust Model for Vocal Pitch Estimation in Polyphonic Musiccited by 27
- 2023The Singing Voice Conversion Challenge 2023cited by 25
- 2025F5R-TTS: Improving Flow-Matching based Text-to-Speech with Group Relative Policy Optimizationcited by 24
- 2024TCSinger: Zero-Shot Singing Voice Synthesis with Style Transfer and Multi-Level Style Controlcited by 20
- 2021Reinforcement Learning for Emotional Text-to-Speech Synthesis with Improved Emotion Discriminabilitycited by 19
- 2024Text-to-Song: Towards Controllable Music Generation Incorporating Vocals and Accompanimentcited by 13
- 2023RMSSinger: Realistic-Music-Score based Singing Voice Synthesiscited by 12
- 2021N-Singer: A Non-Autoregressive Korean Singing Voice Synthesis System for Pronunciation Enhancementcited by 7
- 2022WeSinger: Data-augmented Singing Voice Synthesis with Auxiliary Lossescited by 7
- 2024DLPO: Diffusion Model Loss-Guided Reinforcement Learning for Fine-Tuning Text-to-Speech Diffusion Modelscited by 7
- 2025TCSinger 2: Customizable Multilingual Zero-shot Singing Voice Synthesiscited by 5
- 2025Vevo2: A Unified and Controllable Framework for Speech and Singing Voice Generationcited by 5
- 2021A Survey on Recent Deep Learning-driven Singing Voice Synthesis Systemscited by 3
- 2025CoMelSinger: Discrete Token-Based Zero-Shot Singing Synthesis With Structured Melody Control and Guidancecited by 3
- 2023Leveraging Diverse Semantic-based Audio Pretrained Models for Singing Voice Conversioncited by 2
- 2025SmoothSinger: A Conditional Diffusion Model for Singing Voice Synthesis with Multi-Resolution Architecturecited by 1
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026SoulX-Singer: Towards High-Quality Zero-Shot Singing Voice Synthesiscited by 3
- 2026Advances in GRPO for Generation Models: A Surveycited by 1
- 2026TIGFlow-GRPO: Trajectory Forecasting via Interaction-Aware Flow Matching and Reward-Guided Optimizationcited by 0
- 2026VocalParse: Towards Unified and Scalable Singing Voice Transcription with Large Audio Language Modelscited by 0
- 2026VocalRender: Score-Native Singing Voice Synthesis for Real-World Compositioncited by 0
- 2026VocalRender: Score-Native Singing Voice Synthesis for Real-World Composition30 Jul 2026
- 2026VocalParse: Towards Unified and Scalable Singing Voice Transcription with Large Audio Language Models6 May 2026
- 2026TIGFlow-GRPO: Trajectory Forecasting via Interaction-Aware Flow Matching and Reward-Guided Optimization26 Mar 2026
- 2026Advances in GRPO for Generation Models: A Survey21 Feb 2026
- 2026SoulX-Singer: Towards High-Quality Zero-Shot Singing Voice Synthesis8 Feb 2026
Similar, not cited
Close in wording, no citation either way
- 2025YingMusic-SVC: Real-World Robust Zero-Shot Singing Voice Conversion with Flow-GRPO and Singing-Specific Inductive Biases
- 2025DiTSinger: Scaling Singing Voice Synthesis with Diffusion Transformer and Implicit Alignment
- 2026UniVoice: A Unified Model for Speech and Singing Voice Generation
- 2026MeloDISinger: Melody-Aware & Duration-Preserving Singing Voice Editing with Audio Infilling ↗
- 2026Adapting Diffusion-Based Music Synthesis to Speech and Singing Voice Conversion
- 2026SongCraft: Unified Song Generation and Editing with Reconstructive Learning ↗
- 2025DiffRhythm 2: Efficient and High Fidelity Song Generation via Block Flow Matching
- 2026Mitigating Latent Mismatch in cVAE-Based Singing Voice Synthesis via Flow Matching