# Audio Conditioning for Music Generation via Discrete Bottleneck Features

arXiv: https://arxiv.org/abs/2407.12563 · HTML: https://mlsanity.com/paper/2407.12563/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jul 2024.

## Built on by (most cited first, 6 of 6)

- [MuseControlLite: Multifunctional Music Generation with Lightweight Conditioners](https://mlsanity.com/paper/2506.18729/) (2025)
- [Stable Audio 3](https://mlsanity.com/paper/2605.17991/) (2026)
- [MR-FlowDPO: Multi-Reward Direct Preference Optimization for Flow-Matching Text-to-Music Generation](https://mlsanity.com/paper/2512.10264/) (2025)
- [Low-Resource Guidance for Controllable Latent Audio Diffusion](https://mlsanity.com/paper/2603.04366/) (2026)
- [The Name-Free Gap: Policy-Aware Stylistic Control in Music Generation](https://mlsanity.com/paper/2509.00654/) (2025)
- [Training-Efficient Text-to-Music Generation with State-Space Modeling](https://mlsanity.com/paper/2601.14786/) (2026)

## Abstract (arXiv)

While most music generation models use textual or parametric conditioning (e.g. tempo, harmony, musical genre), we propose to condition a language model based music generation system with audio input. Our exploration involves two distinct strategies. The first strategy, termed textual inversion, leverages a pre-trained text-to-music model to map audio input to corresponding "pseudowords" in the textual embedding space. For the second model we train a music language model from scratch jointly with a text conditioner and a quantized audio feature extractor. At inference time, we can mix textual and audio conditioning and balance them thanks to a novel double classifier free guidance method. We conduct automatic and human studies that validates our approach. We will release the code and we provide music samples on https://musicgenstyle.github.io in order to show the quality of our model.