# Diff2Mix: Controllable Music Mixing via Diffusion Models and Differentiable Audio Effects

arXiv: https://arxiv.org/abs/2608.05442 · HTML: https://mlsanity.com/paper/2608.05442/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Audio, speech and music, one of 14 directions in Diffusion models.
- Builds on 24 of its 42 references that are in the map.
- First posted on arXiv 5 Aug 2026.

## Placed in

- Diffusion models / [Audio, speech and music](https://mlsanity.com/diffusion/audio-speech-and-music/)

## Builds on (most cited first, 24 of 24)

- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation](https://mlsanity.com/paper/2211.06687/) (2022)
- [Neural Autoregressive Flows](https://mlsanity.com/paper/1804.00779/) (2018)
- [DDSP: Differentiable Digital Signal Processing](https://mlsanity.com/paper/2001.04643/) (2020)
- [Multi-Singer: Fast Multi-Singer Singing Voice Vocoder With A Large-Scale Corpus](https://mlsanity.com/paper/2112.10358/) (2021)
- [Moisesdb: A dataset for source separation beyond 4-stems](https://mlsanity.com/paper/2307.15913/) (2023)
- [Automatic multitrack mixing with a differentiable mixing console of neural audio effects](https://mlsanity.com/paper/2010.10291/) (2020)
- [KAD: No More FAD! An Effective and Efficient Evaluation Metric for Audio Generation](https://mlsanity.com/paper/2502.15602/) (2025)
- [Style Transfer of Audio Effects with Differentiable Signal Processing](https://mlsanity.com/paper/2207.08759/) (2022)
- [Music Mixing Style Transfer: A Contrastive Learning Approach to Disentangle Audio Effects](https://mlsanity.com/paper/2211.02247/) (2022)
- [Filtered Noise Shaping for Time Domain Room Impulse Response Estimation From Reverberant Speech](https://mlsanity.com/paper/2107.07503/) (2021)
- [InverSynth: Deep Estimation of Synthesizer Parameter Configurations from Audio Signals](https://mlsanity.com/paper/1812.06349/) (2018)
- [Automatic music mixing with deep learning and out-of-domain data](https://mlsanity.com/paper/2208.11428/) (2022)
- [Text2FX: Harnessing CLAP Embeddings for Text-Guided Audio Effects](https://mlsanity.com/paper/2409.18847/) (2024)
- [Direct design of biquad filter cascades with deep learning by sampling random polynomials](https://mlsanity.com/paper/2110.03691/) (2021)
- [ST-ITO: Controlling Audio Effects for Style Transfer with Inference-Time Optimization](https://mlsanity.com/paper/2410.21233/) (2024)
- [End-to-end Music Remastering System Using Self-supervised and Adversarial Training](https://mlsanity.com/paper/2202.08520/) (2022)
- [Diff-MST: Differentiable Mixing Style Transfer](https://mlsanity.com/paper/2407.08889/) (2024)
- [Audio synthesizer inversion in symmetric parameter spaces with approximately equivariant flow matching](https://mlsanity.com/paper/2506.07199/) (2025)
- [Fx-Encoder++: Extracting Instrument-Wise Audio Effects Representations from Mixtures](https://mlsanity.com/paper/2507.02273/) (2025)
- [Automatic Music Mixing using a Generative Model of Effect Embeddings](https://mlsanity.com/paper/2511.08040/) (2025)
- [LLM2Fx-Tools: Tool Calling For Music Post-Production](https://mlsanity.com/paper/2512.01559/) (2025)

## Abstract (arXiv)

Automatic music mixing aims to combine multitrack recordings into a balanced and coherent musical piece. Because the content of different songs and the subjective preferences of mixing engineers jointly shape the final outcome, a practical system should deliver well-balanced mixes while allowing for controllable stylistic variation. However, most existing methods treat automatic mixing and mixing style control as separate tasks, making it difficult for a single system to produce high-quality mixes while remaining editable and style-aware. To address this limitation, this paper presents Diff2Mix, a generative automatic mixing system based on diffusion models and a differentiable mixing console. This system offers two levels of optional user control: a reference audio enables overall production style control, and the differentiable mixing console provides explicit audio effects parameters for interpretability and fine-grained optimization. We demonstrate our system's competitive performance through both objective and subjective evaluations in terms of mixing quality and control ability. We provide code and audio samples at our project page https://zys711.github.io/Diff2Mix .