mlsanityData to 14 Oct 2026

Adversarial Permutation Invariant Training for Universal Sound Separation

arXiv 2210.12108 ↗First posted 21 Oct 2022cs.SD cs.LG eess.AScited by 8

Not on the map. Why?

Abstract (arXiv)

Universal sound separation consists of separating mixes with arbitrary sounds of different types, and permutation invariant training (PIT) is used to train source agnostic models that do so. In this work, we complement PIT with adversarial losses but find it challenging with the standard formulation used in speech source separation. We overcome this challenge with a novel I-replacement context-based adversarial loss, and by training with multiple discriminators. Our experiments show that by simply improving the loss (keeping the same model and dataset) we obtain a non-negligible improvement of 1.4 dB SI-SNRi in the reverberant FUSS dataset. We also find adversarial PIT to be effective at reducing spectral holes, ubiquitous in mask-based separation models, which highlights the potential relevance of adversarial losses for source separation.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 22 of its 32 references that are in the map. First posted on arXiv 21 Oct 2022.

Builds on

22 of 32 references.

References · 22

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  2. 2022Generative Adversarial Networkscited by 5,627
  3. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  4. 2018Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separationcited by 406
  5. 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 354
  6. 2020FSD50K: An Open Dataset of Human-Labeled Sound Eventscited by 342
  7. 2017Geometric GANcited by 233
  8. 2016Permutation Invariant Training of Deep Models for Speaker-Independent Multi-talker Speech Separationcited by 191
  9. 2017SEGAN: Speech Enhancement Generative Adversarial Networkcited by 166
  10. 2019High Fidelity Speech Synthesis with Adversarial Networkscited by 92
  11. 2019Universal Sound Separationcited by 41
  12. 2018Differentiable Consistency Constraints for Improved Deep Speech Enhancementcited by 35
10 more
  1. 2020Upsampling artifacts in neural audio synthesiscited by 28
  2. 2020What's All the FUSS About Free Universal Sound Separation Data?cited by 22
  3. 2017Generative Adversarial Source Separationcited by 19
  4. 2018End-to-end music source separation: is it possible in the waveform domain?cited by 19
  5. 2020Unsupervised Audio Source Separation using Generative Priorscited by 17
  6. 2022On loss functions and evaluation metrics for music source separationcited by 17
  7. 2020An empirical study of Conv-TasNetcited by 15
  8. 2017Adversarial Semi-Supervised Audio Source Separation applied to Singing Voice Extractioncited by 11
  9. 2021Sparse, Efficient, and Semantic Mixture Invariant Training: Taming In-the-Wild Unsupervised Sound Separationcited by 9
  10. 2021On permutation invariant training for speech source separationcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 2; 2025: 2; 2026: 1.

8 papers.

  1. 2023Multi-Source Diffusion Models for Simultaneous Music Generation and Separationcited by 63
  2. 2025Fast Text-to-Audio Generation with Adversarial Post-Trainingcited by 14
  3. 2024Simultaneous Music Separation and Generation Using Multi-Track Latent Diffusion Modelscited by 13
  4. 2023Latent Autoregressive Source Separationcited by 12
  5. 2023GASS: Generalizing Audio Source Separation with Large-scale Datacited by 6
  6. 2024Improving Music Source Separation with Diffusion and Consistency Refinementcited by 1
  7. 2025MARS-Sep: Multimodal-Aligned Reinforced Sound Separationcited by 1
  8. 2026Iterative Audio Separation with Mixture Consistency via MIMO Model Extensioncited by 0