mlsanityData to 7 Oct 2026

SingVisio: Visual Analytics of Diffusion Model for Singing Voice Conversion

arXiv 2402.12660 ↗First posted 20 Feb 2024cs.SD cs.HC eess.AScited by 2

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

In this study, we present SingVisio, an interactive visual analysis system that aims to explain the diffusion model used in singing voice conversion. SingVisio provides a visual display of the generation process in diffusion models, showcasing the step-by-step denoising of the noisy spectrum and its transformation into a clean spectrum that captures the desired singer's timbre. The system also facilitates side-by-side comparisons of different conditions, such as source content, melody, and target timbre, highlighting the impact of these conditions on the diffusion generation process and resulting conversions. Through comparative and comprehensive evaluations, SingVisio demonstrates its effectiveness in terms of system design, functionality, explainability, and user-friendliness. It offers users of various backgrounds valuable learning experiences and insights into the diffusion model for singing voice conversion.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 38 of its 59 references that are in the map. First posted on arXiv 20 Feb 2024.

Builds on

38 of 59 references.

Closest ancestors · 9

Same direction, newest first

  1. 2024CoMoSVC: Consistency Model-based Singing Voice Conversion3 Jan
  2. 2023Diff-HierVC: Diffusion-based Hierarchical Voice Conversion with Robust Pitch Generation and Masked Prior for Zero-shot Speaker Adaptation8 Nov
  3. 2023Make-An-Audio 2: Temporal-Enhanced Text-to-Audio Generation29 May
  4. 2023NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers18 Apr
  5. 2023AUDIT: Audio Editing by Following Instructions with Latent Diffusion Models3 Apr
  6. 2021Diffusion-Based Voice Conversion with Fast Maximum Likelihood Sampling Scheme28 Sep
  7. 2021DiffSVC: A Diffusion Probabilistic Model for Singing Voice Conversion28 May
  8. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech13 May
  9. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesis21 Sep

Other references · 29

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2022Generative Adversarial Networkscited by 5,603
  3. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  4. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  5. 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,543
  6. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
  7. 2023Pix2Video: Video Editing using Image Diffusioncited by 318
  8. 2023Text-to-image Diffusion Models in Generative AI: A Surveycited by 264
  9. 2021DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanismcited by 154
  10. 2018Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontierscited by 117
  11. 2020An Overview of Voice Conversion and its Challenges: From Statistical Modeling to Deep Learningcited by 111
  12. 2022ContentVec: An Improved Self-Supervised Speech Representation by Disentangling Speakerscited by 73
17 more
  1. 2021Multi-Singer: Fast Multi-Singer Singing Voice Vocoder With A Large-Scale Corpuscited by 60
  2. 2020CNN Explainer: Learning Convolutional Neural Networks with Interactive Visualizationcited by 57
  3. 2022Opencpop: A High-Quality Open Source Chinese Popular Song Corpus for Singing Voice Synthesiscited by 53
  4. 2016Towards Better Analysis of Deep Convolutional Neural Networkscited by 51
  5. 2023AttentionViz: A Global View of Transformer Attentioncited by 50
  6. 2018GAN Lab: Understanding Complex Deep Generative Models using Interactive Visual Experimentationcited by 40
  7. 2021M2Lens: Visualizing and Explaining Multimodal Models for Sentiment Analysiscited by 29
  8. 2023The Singing Voice Conversion Challenge 2023cited by 25
  9. 2023Amphion: An Open-Source Audio, Music and Speech Generation Toolkitcited by 25
  10. 2020FastSVC: Fast Cross-Domain Singing Voice Conversion with Feature-wise Linear Modulationcited by 21
  11. 2019Singing Voice Conversion with Disentangled Representations of Singer and Vocal Technique Using Variational Autoencoderscited by 18
  12. 2024Explaining generative diffusion models via visual analysis for interpretable decision-making processcited by 15
  13. 2019Unsupervised Singing Voice Conversioncited by 14
  14. 2022A Comparative Study of Self-supervised Speech Representation Based Voice Conversioncited by 9
  15. 2019Singing voice conversion with non-parallel datacited by 8
  16. 2017Adversarial-Playground: A Visualization Suite Showing How Adversarial Examples Fool Deep Learningcited by 7
  17. 2022Robust One-Shot Singing Voice Conversioncited by 5

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 1.

2 papers.

  1. 2025MoEdit: On Learning Quantity Perception for Multi-object Image Editingcited by 7
  2. 2024LHQ-SVC: Lightweight and High Quality Singing Voice Conversion Modelingcited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2024SINGER: Vivid Audio-driven Singing Video Generation with Multi-scale Spectral Diffusion Model
  2. 2024RDSinger: Reference-based Diffusion Network for Singing Voice Synthesis
  3. 2026Adapting Diffusion-Based Music Synthesis to Speech and Singing Voice Conversion
  4. 2025Generating Separated Singing Vocals Using a Diffusion Model Conditioned on Music Mixtures
  5. 2025SmoothSinger: A Conditional Diffusion Model for Singing Voice Synthesis with Multi-Resolution Architecture
  6. 2026Diff-VS: Efficient Audio-Aware Diffusion U-Net for Vocals Separation
  7. 2023HiddenSinger: High-Quality Singing Voice Synthesis via Neural Audio Codec and Latent Diffusion Models
  8. 2025Efficient and Fast Generative-Based Singing Voice Separation using a Latent Diffusion Model