mlsanityData to 14 Oct 2026

Hybrid Spectrogram and Waveform Source Separation

arXiv 2111.03600 ↗First posted 5 Nov 2021eess.AS cs.SD stat.MLcited by 91

Not on the map. Why?

Abstract (arXiv)

Source separation models either work on the spectrogram or waveform domain. In this work, we show how to perform end-to-end hybrid source separation, letting the model decide which domain is best suited for each source, and even combining both. The proposed hybrid version of the Demucs architecture won the Music Demixing Challenge 2021 organized by Sony. This architecture also comes with additional improvements, such as compressed residual branches, local attention or singular value regularization. Overall, a 1.4 dB improvement of the Signal-To-Distortion (SDR) was observed across all sources as measured on the MusDB HQ dataset, an improvement confirmed by human subjective evaluation, with an overall quality rated at 2.83 out of 5 (2.36 for the non hybrid Demucs), and absence of contamination at 3.04 (against 2.37 for the non hybrid Demucs and 2.44 for the second ranking model submitted at the competition).

Key facts. Cited by 91 papers in the map (data through 14 Oct 2026). Builds on 23 of its 34 references that are in the map. First posted on arXiv 5 Nov 2021.

Builds on

23 of 34 references.

References · 23

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2016Layer Normalizationcited by 5,669
  4. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  5. 2015Multi-Scale Context Aggregation by Dilated Convolutionscited by 1,133
  6. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,103
  7. 2021Going deeper with Image Transformerscited by 764
  8. 2016Language Modeling with Gated Convolutional Networkscited by 702
  9. 2018Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separationcited by 406
  10. 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 354
  11. 2017Spectral Norm Regularization for Improving the Generalizability of Deep Learningcited by 182
  12. 2019Dual-path RNN: efficient long sequence modeling for time-domain single-channel speech separationcited by 182
11 more
  1. 2018Wave-U-Net: A Multi-Scale Neural Network for End-to-End Audio Source Separationcited by 125
  2. 2019Music Source Separation in the Waveform Domaincited by 99
  3. 2021Decoupling Magnitude and Phase Estimation with Deep ResUNet for Music Source Separationcited by 48
  4. 2018The 2018 Signal Separation Evaluation Campaigncited by 41
  5. 2021Music Demixing Challenge 2021cited by 38
  6. 2020Upsampling artifacts in neural audio synthesiscited by 28
  7. 2020D3Net: Densely connected multidilated DenseNet for music source separationcited by 24
  8. 2020PoCoNet: Better Speech Enhancement with Frequency-Positional Embeddings, Semi-Supervised Conversational Data, and Biased Losscited by 23
  9. 2020All for One and One for All: Improving Music Separation by Bridging Networkscited by 20
  10. 2018End-to-end music source separation: is it possible in the waveform domain?cited by 19
  11. 2020LaSAFT: Latent Source Attentive Frequency Transformation for Conditioned Source Separationcited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 12; 2023: 18; 2024: 22; 2025: 25; 2026: 14.

91 papers.

Sort:

By area:

  1. 2022Music Source Separation with Band-split RNNcited by 70
  2. 2023Multi-Source Diffusion Models for Simultaneous Music Generation and Separationcited by 63
  3. 2025LeVo: High-Quality Song Generation with Multi-Preference Alignmentcited by 25
  4. 2024UniMuMo: Unified Text, Music and Motion Generationcited by 19
  5. 2022Music Mixing Style Transfer: A Contrastive Learning Approach to Disentangle Audio Effectscited by 18
  6. 2022AERO: Audio Super Resolution in the Spectral Domaincited by 17
  7. 2023JEN-1 Composer: A Unified Framework for High-Fidelity Multi-Track Music Generationcited by 17
  8. 2024MuChin: A Chinese Colloquial Description Benchmark for Evaluating Language Models in the Field of Musiccited by 16
  9. 2024IndicVoices-R: Unlocking a Massive Multilingual Multi-speaker Speech Corpus for Scaling Indian TTScited by 16
  10. 2025Analyzable Chain-of-Musical-Thought Prompting for High-Fidelity Music Generationcited by 14
  11. 2024Simultaneous Music Separation and Generation Using Multi-Track Latent Diffusion Modelscited by 13
  12. 2023Latent Autoregressive Source Separationcited by 12
38 more
  1. 2024SongCreator: Lyrics-based Universal Song Generationcited by 12
  2. 2023Songs Across Borders: Singable and Controllable Neural Lyric Translationcited by 11
  3. 2024MusiConGen: Rhythm and Chord Control for Transformer-Based Text-to-Music Generationcited by 11
  4. 2025JAM: A Tiny Flow-based Song Generator with Fine-grained Controllability and Aesthetic Alignmentcited by 9
  5. 2022Music Separation Enhancement with Generative Modelingcited by 8
  6. 2023A Generalized Bandsplit Neural Network for Cinematic Audio Source Separationcited by 8
  7. 2023Evaluating Self-supervised Speech Models on a Taiwanese Hokkien Corpuscited by 7
  8. 2024A Stem-Agnostic Single-Decoder System for Music Source Separation Beyond Four Stemscited by 7
  9. 2022Implicit Neural Spatial Filtering for Multichannel Source Separation in the Waveform Domaincited by 6
  10. 2023Benchmarks and leaderboards for sound demixing taskscited by 6
  11. 2025SonicMaster: Towards Controllable All-in-One Music Restoration and Masteringcited by 6
  12. 2026UnityShots: Memory-Driven Multi-Shot Audio-Video Generation with Boundary-Aware Gatingcited by 6
  13. 2022JaCappella Corpus: A Japanese a Cappella Vocal Ensemble Corpuscited by 5
  14. 2025Recognizing Ornaments in Vocal Indian Art Music with Active Annotationcited by 5
  15. 2025Learning to Highlight Audio by Watching Moviescited by 5
  16. 2025Music Source Restorationcited by 5
  17. 2022Distortion Audio Effects: Learning How to Recover the Clean Signalcited by 4
  18. 2022A differentiable short-time Fourier transform with respect to the window lengthcited by 4
  19. 2023Sound Demixing Challenge 2023 Music Demixing Track Technical Report: TFC-TDF-UNet v3cited by 4
  20. 2023Toward Deep Drum Source Separationcited by 4
  21. 2024SynthSOD: Developing an Heterogeneous Dataset for Orchestra Music Source Separationcited by 4
  22. 2025MGE-LDM: Joint Latent Diffusion for Simultaneous Music Generation and Source Extractioncited by 4
  23. 2022MedleyVox: An Evaluation Dataset for Multiple Singing Voices Separationcited by 3
  24. 2023The ICASSP SP Cadenza Challenge: Music Demixing/Remixing for Hearing Aidscited by 3
  25. 2023The First Cadenza Signal Processing Challenge: Improving Music for Those With a Hearing Losscited by 3
  26. 2024Real-time Low-latency Music Source Separation using Hybrid Spectrogram-TasNetcited by 3
  27. 2024From Real to Cloned Singer Identificationcited by 3
  28. 2025The Inverse Drum Machine: Source Separation Through Joint Transcription and Analysis-by-Synthesiscited by 3
  29. 2025Zero-Shot Text-to-Speech for Vietnamesecited by 3
  30. 2022Deep Audio Waveform Priorcited by 2
  31. 2024Machine Perceptual Quality: Evaluating the Impact of Severe Lossy Compression on Audio and Image Modelscited by 2
  32. 2024MuDiT & MuSiT: Alignment with Colloquial Expression in Description-to-Song Generationcited by 2
  33. 2024A Two-Stage Band-Split Mamba-2 Network For Music Separationcited by 2
  34. 2025Sanidha: A Studio Quality Multi-Modal Dataset for Carnatic Musiccited by 2
  35. 2025NeuroAMP: A Novel End-to-end General Purpose Deep Neural Amplifier for Personalized Hearing Aidscited by 2
  36. 2025Score-informed Music Source Separation: Improving Synthetic-to-real Generalization in Classical Musiccited by 2
  37. 2025Efficient and Fast Generative-Based Singing Voice Separation using a Latent Diffusion Modelcited by 2
  38. 2026DARC: Drum accompaniment generation with fine-grained rhythm controlcited by 2

50 most cited of 91 All 91 (JSON)