mlsanityData to 7 Oct 2026

Hybrid Transformers for Music Source Separation

arXiv 2211.08553 ↗First posted 15 Nov 2022eess.AS cs.SDcited by 134

Not on the map. Why?

Abstract (arXiv)

A natural question arising in Music Source Separation (MSS) is whether long range contextual information is useful, or whether local acoustic features are sufficient. In other fields, attention based Transformers have shown their ability to integrate information over long sequences. In this work, we introduce Hybrid Transformer Demucs (HT Demucs), an hybrid temporal/spectral bi-U-Net based on Hybrid Demucs, where the innermost layers are replaced by a cross-domain Transformer Encoder, using self-attention within one domain, and cross-attention across domains. While it performs poorly when trained only on MUSDB, we show that it outperforms Hybrid Demucs (trained on the same data) by 0.45 dB of SDR when using 800 extra training songs. Using sparse attention kernels to extend its receptive field, and per source fine-tuning, we achieve state-of-the-art results on MUSDB with extra training data, with 9.20 dB of SDR.

Key facts. Cited by 134 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 Nov 2022.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 20; 2024: 38; 2025: 40; 2026: 36.

134 papers.

Sort:

By area:

  1. 2023MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Trainingcited by 175
  2. 2025Step-Audio: Unified Understanding and Generation in Intelligent Speech Interactioncited by 83
  3. 2024EARS: An Anechoic Fullband Speech Dataset Benchmarked for Speech Enhancement and Dereverberationcited by 62
  4. 2024VidMuse: A Simple Video-to-Music Generation Framework with Long-Short-Term Modelingcited by 39
  5. 2023MARBLE: Music Audio Representation Benchmark for Universal Evaluationcited by 38
  6. 2023Moisesdb: A dataset for source separation beyond 4-stemscited by 37
  7. 2024DASB - Discrete Audio and Speech Benchmarkcited by 32
  8. 2025SongGen: A Single Stage Auto-regressive Transformer for Text-to-Song Generationcited by 29
  9. 2025LeVo: High-Quality Song Generation with Multi-Preference Alignmentcited by 23
  10. 2023SingFake: Singing Voice Deepfake Detectioncited by 21
  11. 2024SONICS: Synthetic Or Not -- Identifying Counterfeit Songscited by 21
  12. 2025SongBloom: Coherent Song Generation via Interleaved Autoregressive Sketching and Diffusion Refinementcited by 21
38 more
  1. 2023JEN-1 Composer: A Unified Framework for High-Fidelity Multi-Track Music Generationcited by 17
  2. 2025X-Streamer: Unified Human World Modeling with Audiovisual Interactioncited by 16
  3. 2023Loop Copilot: Conducting AI Ensembles for Music Generation and Iterative Editingcited by 15
  4. 2024MuChin: A Chinese Colloquial Description Benchmark for Evaluating Language Models in the Field of Musiccited by 15
  5. 2024VMAS: Video-to-Music Generation via Semantic Alignment in Web Music Videoscited by 14
  6. 2024MMTrail: A Multimodal Trailer Video Dataset with Language and Music Descriptionscited by 13
  7. 2025Analyzable Chain-of-Musical-Thought Prompting for High-Fidelity Music Generationcited by 13
  8. 2024A Diffusion-Based Generative Equalizer for Music Restorationcited by 12
  9. 2024SongCreator: Lyrics-based Universal Song Generationcited by 12
  10. 2024MusiConGen: Rhythm and Chord Control for Transformer-Based Text-to-Music Generationcited by 11
  11. 2024FunnyNet-W: Multimodal Learning of Funny Moments in Videos in the Wildcited by 9
  12. 2023A Generalized Bandsplit Neural Network for Cinematic Audio Source Separationcited by 8
  13. 2024OmniSep: Unified Omni-Modality Sound Separation with Query-Mixupcited by 8
  14. 2024A Stem-Agnostic Single-Decoder System for Music Source Separation Beyond Four Stemscited by 7
  15. 2025TTSDS2: Resources and Benchmark for Evaluating Human-Quality Text to Speech Systemscited by 7
  16. 2025AutoMV: An Automatic Multi-Agent System for Music Video Generationcited by 7
  17. 2023Benchmarks and leaderboards for sound demixing taskscited by 6
  18. 2023GASS: Generalizing Audio Source Separation with Large-scale Datacited by 6
  19. 2023Novel-View Acoustic Synthesis from 3D Reconstructed Roomscited by 6
  20. 2024Music Foundation Model as Generic Booster for Music Downstream Taskscited by 6
  21. 2024Multimodal Music Generation with Explicit Bridges and Retrieval Augmentationcited by 6
  22. 2025FusionAudio-1.2M: Towards Fine-grained Audio Captioning with Multimodal Contextual Fusioncited by 6
  23. 2024Lyrics Transcription for Humans: A Readability-Aware Benchmarkcited by 5
  24. 2025SafeSpeech: Robust and Universal Voice Protection Against Malicious Speech Synthesiscited by 5
  25. 2025Music Source Restorationcited by 5
  26. 2025SongPrep: A Preprocessing Framework and End-to-end Model for Full-song Structure Parsing and Lyrics Transcriptioncited by 5
  27. 2026AVControl: Efficient Framework for Training Audio-Visual Controlscited by 5
  28. 2023Sound Demixing Challenge 2023 Music Demixing Track Technical Report: TFC-TDF-UNet v3cited by 4
  29. 2024SynthSOD: Developing an Heterogeneous Dataset for Orchestra Music Source Separationcited by 4
  30. 2026IndexTTS 2.5 Technical Reportcited by 4
  31. 2024Real-time Low-latency Music Source Separation using Hybrid Spectrogram-TasNetcited by 3
  32. 2024From Real to Cloned Singer Identificationcited by 3
  33. 2024MusicLIME: Explainable Multimodal Music Understandingcited by 3
  34. 2025Diffusion based Text-to-Music Generation with Global and Local Text based Conditioningcited by 3
  35. 2025Zero-Shot Text-to-Speech for Vietnamesecited by 3
  36. 2026Laya: A LeJEPA Approach to EEG via Latent Prediction over Reconstructioncited by 3
  37. 2023TorchBench: Benchmarking PyTorch with High API Surface Coveragecited by 2
  38. 2023Jam-ALT: A Formatting-Aware Lyrics Transcription Benchmarkcited by 2

50 most cited of 134 All 134 (JSON)