mlsanityData to 14 Oct 2026

Song Data Cleansing for End-to-End Neural Singer Diarization Using Neural Analysis and Synthesis Framework

arXiv 2406.16315 ↗First posted 24 Jun 2024eess.AS cs.CL cs.SDcited by 0

Not on the map. Why?

Abstract (arXiv)

We propose a data cleansing method that utilizes a neural analysis and synthesis (NANSY++) framework to train an end-to-end neural diarization model (EEND) for singer diarization. Our proposed model converts song data with choral singing which is commonly contained in popular music and unsuitable for generating a simulated dataset to the solo singing data. This cleansing is based on NANSY++, which is a framework trained to reconstruct an input non-overlapped audio signal. We exploit the pre-trained NANSY++ to convert choral singing into clean, non-overlapped audio. This cleansing process mitigates the mislabeling of choral singing to solo singing and helps the effective training of EEND models even when the majority of available song data contains choral singing sections. We experimentally evaluated the EEND model trained with a dataset using our proposed method using annotated popular duet songs. As a result, our proposed method improved 14.8 points in diarization error rate.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 31 references that are in the map. First posted on arXiv 24 Jun 2024.

Builds on

22 of 31 references.

References · 22

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  3. 2019pyannote.audio: neural building blocks for speaker diarizationcited by 181
  4. 2018Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotroncited by 173
  5. 2022Hybrid Transformers for Music Source Separationcited by 134
  6. 2023LibriTTS-R: A Restored Multi-Speaker Text-to-Speech Corpuscited by 101
  7. 2017Multi-talker Speech Separation with Utterance-level Permutation Invariant Training of Deep Recurrent Neural Networkscited by 98
  8. 2021Neural Analysis and Synthesis: Reconstructing Speech from Self-Supervised Representationscited by 81
  9. 2019End-to-End Neural Speaker Diarization with Self-attentioncited by 77
  10. 2020Wavesplit: End-to-End Speech Separation by Speaker Clusteringcited by 75
  11. 2022ContentVec: An Improved Self-Supervised Speech Representation by Disentangling Speakerscited by 74
  12. 2020Bayesian HMM clustering of x-vector sequences (VBx) in speaker diarization: theory, implementation and analysis on standard taskscited by 73
10 more
  1. 2020End-to-End Speaker Diarization for an Unknown Number of Speakers with Encoder-Decoder Based Attractorscited by 71
  2. 2020Target-Speaker Voice Activity Detection: a Novel Approach for Multi-Speaker Diarization in a Dinner Party Scenariocited by 68
  3. 2022NANSY++: Unified Voice Synthesis with Neural Analysis and Synthesiscited by 38
  4. 2020Integrating end-to-end neural and clustering-based diarization: Getting the best of both worldscited by 35
  5. 2022From Simulated Mixtures to Simulated Conversations as Training Data for End-to-End Neural Diarizationcited by 20
  6. 2021Auxiliary Loss of Transformer with Residual Connection for End-to-End Speaker Diarizationcited by 7
  7. 2018Vocal melody extraction using patch-based CNNcited by 6
  8. 2020Multiple F0 Estimation in Vocal Ensembles using Convolutional Neural Networkscited by 6
  9. 2022Period VITS: Variational Inference with Explicit Pitch Modeling for End-to-end Emotional Speech Synthesiscited by 6
  10. 2021Semi-Supervised Training with Pseudo-Labeling for End-to-End Neural Diarizationcited by 3

Built on by

Not cited yet.