# Audio style transfer

arXiv: https://arxiv.org/abs/1710.11385 · HTML: https://mlsanity.com/paper/1710.11385/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Oct 2017.

## Built on by (most cited first, 15 of 15)

- [HateMM: A Multi-Modal Dataset for Hate Video Classification](https://mlsanity.com/paper/2305.03915/) (2023)
- [Going In Style: Audio Backdoors Through Stylistic Transformations](https://mlsanity.com/paper/2211.03117/) (2022)
- [Prompt-guided Precise Audio Editing with Diffusion Models](https://mlsanity.com/paper/2406.04350/) (2024)
- [Self-Supervised VQ-VAE for One-Shot Music Style Transfer](https://mlsanity.com/paper/2102.05749/) (2021)
- [Music Style Transfer with Time-Varying Inversion of Diffusion Models](https://mlsanity.com/paper/2402.13763/) (2024)
- [Towards a Robust Framework for Multimodal Hate Detection: A Study on Video vs. Image-based Content](https://mlsanity.com/paper/2502.07138/) (2025)
- [StyleTime: Style Transfer for Synthetic Time Series Generation](https://mlsanity.com/paper/2209.11306/) (2022)
- [Towards Evaluating the Robustness of Automatic Speech Recognition Systems via Audio Style Transfer](https://mlsanity.com/paper/2405.09470/) (2024)
- [CAESynth: Real-Time Timbre Interpolation and Pitch Control with Conditional Autoencoders](https://mlsanity.com/paper/2111.05174/) (2021)
- [Repurposing Image Diffusion Models for Training-Free Music Style Transfer on Mel-spectrograms](https://mlsanity.com/paper/2411.15913/) (2024)
- [Revisiting Your Memory: Reconstruction of Affect-Contextualized Memory via EEG-guided Audiovisual Generation](https://mlsanity.com/paper/2412.05296/) (2024)
- [Environment Transfer for Distributed Systems](https://mlsanity.com/paper/2101.01863/) (2021)
- [Emotion Transfer Using Vector-Valued Infinite Task Learning](https://mlsanity.com/paper/2102.05075/) (2021)
- [Voice Aging with Audio-Visual Style Transfer](https://mlsanity.com/paper/2110.02411/) (2021)
- ["Melatonin": A Case Study on AI-induced Musical Style](https://mlsanity.com/paper/2208.08968/) (2022)

## Abstract (arXiv)

'Style transfer' among images has recently emerged as a very active research topic, fuelled by the power of convolution neural networks (CNNs), and has become fast a very popular technology in social media. This paper investigates the analogous problem in the audio domain: How to transfer the style of a reference audio signal to a target audio content? We propose a flexible framework for the task, which uses a sound texture model to extract statistics characterizing the reference audio style, followed by an optimization-based audio texture synthesis to modify the target content. In contrast to mainstream optimization-based visual transfer method, the proposed process is initialized by the target content instead of random noise and the optimized loss is only about texture, not structure. These differences proved key for audio style transfer in our experiments. In order to extract features of interest, we investigate different architectures, whether pre-trained on other tasks, as done in image style transfer, or engineered based on the human auditory system. Experimental results on different types of audio signal confirm the potential of the proposed approach.