# Music Separation Enhancement with Generative Modeling

arXiv: https://arxiv.org/abs/2208.12387 · HTML: https://mlsanity.com/paper/2208.12387/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 63 references that are in the map.
- First posted on arXiv 26 Aug 2022.

## Builds on (most cited first, 30 of 39)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [SDR - half-baked or well done?](https://mlsanity.com/paper/1811.02508/) (2018)
- [Autoencoding beyond pixels using a learned similarity metric](https://mlsanity.com/paper/1512.09300/) (2015)
- [MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis](https://mlsanity.com/paper/1910.06711/) (2019)
- [Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram](https://mlsanity.com/paper/1910.11480/) (2019)
- [Adversarial Audio Synthesis](https://mlsanity.com/paper/1802.04208/) (2018)
- [Deep clustering: Discriminative embeddings for segmentation and separation](https://mlsanity.com/paper/1508.04306/) (2015)
- [Real Time Speech Enhancement in the Waveform Domain](https://mlsanity.com/paper/2006.12847/) (2020)
- [SEGAN: Speech Enhancement Generative Adversarial Network](https://mlsanity.com/paper/1703.09452/) (2017)
- [Wave-U-Net: A Multi-Scale Neural Network for End-to-End Audio Source Separation](https://mlsanity.com/paper/1806.03185/) (2018)
- [GANSynth: Adversarial Neural Audio Synthesis](https://mlsanity.com/paper/1902.08710/) (2019)
- [TasNet: time-domain audio separation network for real-time, single-channel speech separation](https://mlsanity.com/paper/1711.00541/) (2017)
- [MetricGAN+: An Improved Version of MetricGAN for Speech Enhancement](https://mlsanity.com/paper/2104.03538/) (2021)
- [MetricGAN: Generative Adversarial Networks based Black-box Metric Scores Optimization for Speech Enhancement](https://mlsanity.com/paper/1905.04874/) (2019)
- [Music Source Separation in the Waveform Domain](https://mlsanity.com/paper/1911.13254/) (2019)
- [High Fidelity Speech Synthesis with Adversarial Networks](https://mlsanity.com/paper/1909.11646/) (2019)
- [Hybrid Spectrogram and Waveform Source Separation](https://mlsanity.com/paper/2111.03600/) (2021)
- [UnivNet: A Neural Vocoder with Multi-Resolution Spectrogram Discriminators for High-Fidelity Waveform Generation](https://mlsanity.com/paper/2106.07889/) (2021)
- [Neural source-filter waveform models for statistical parametric speech synthesis](https://mlsanity.com/paper/1904.12088/) (2019)
- [The 2018 Signal Separation Evaluation Campaign](https://mlsanity.com/paper/1804.06267/) (2018)
- [HiFi-GAN: High-Fidelity Denoising and Dereverberation Based on Speech Deep Features in Adversarial Networks](https://mlsanity.com/paper/2006.05694/) (2020)
- [Music Demixing Challenge 2021](https://mlsanity.com/paper/2108.13559/) (2021)
- [Upsampling artifacts in neural audio synthesis](https://mlsanity.com/paper/2010.14356/) (2020)
- [D3Net: Densely connected multidilated DenseNet for music source separation](https://mlsanity.com/paper/2010.01733/) (2020)
- [End-to-end music source separation: is it possible in the waveform domain?](https://mlsanity.com/paper/1810.12187/) (2018)
- [Simultaneous Separation and Transcription of Mixtures with Multiple Polyphonic and Percussive Instruments](https://mlsanity.com/paper/1910.12621/) (2019)
- [On loss functions and evaluation metrics for music source separation](https://mlsanity.com/paper/2202.07968/) (2022)
- [A Two-Stage U-Net for High-Fidelity Denoising of Historical Recordings](https://mlsanity.com/paper/2202.08702/) (2022)

## Built on by (most cited first, 8 of 8)

- [TokenSplit: Using Discrete Speech Representations for Direct, Refined, and Transcript-Conditioned Speech Separation and Recognition](https://mlsanity.com/paper/2308.10415/) (2023)
- [JaCappella Corpus: A Japanese a Cappella Vocal Ensemble Corpus](https://mlsanity.com/paper/2211.16028/) (2022)
- [Exploiting Time-Frequency Conformers for Music Audio Enhancement](https://mlsanity.com/paper/2308.12599/) (2023)
- [Distribution Preserving Source Separation With Time Frequency Predictive Models](https://mlsanity.com/paper/2303.05896/) (2023)
- [Improving Music Source Separation with Diffusion and Consistency Refinement](https://mlsanity.com/paper/2412.06965/) (2024)
- [Generating Separated Singing Vocals Using a Diffusion Model Conditioned on Music Mixtures](https://mlsanity.com/paper/2511.21342/) (2025)
- [A Knowledge-Driven Approach to Music Segmentation, Music Source Separation and Cinematic Audio Source Separation](https://mlsanity.com/paper/2602.21476/) (2026)
- [DTT-BSR+: A Generative-Regression Cascade for Music Source Restoration](https://mlsanity.com/paper/2606.24127/) (2026)

## Abstract (arXiv)

Despite phenomenal progress in recent years, state-of-the-art music separation systems produce source estimates with significant perceptual shortcomings, such as adding extraneous noise or removing harmonics. We propose a post-processing model (the Make it Sound Good (MSG) post-processor) to enhance the output of music source separation systems. We apply our post-processing model to state-of-the-art waveform-based and spectrogram-based music source separators, including a separator unseen by MSG during training. Our analysis of the errors produced by source separators shows that waveform models tend to introduce more high-frequency noise, while spectrogram models tend to lose transients and high frequency content. We introduce objective measures to quantify both kinds of errors and show MSG improves the source reconstruction of both kinds of errors. Crowdsourced subjective evaluations demonstrate that human listeners prefer source estimates of bass and drums that have been post-processed by MSG.