# Conditional End-to-End Audio Transforms

arXiv: https://arxiv.org/abs/1804.00047 · HTML: https://mlsanity.com/paper/1804.00047/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Mar 2018.

## Built on by (most cited first, 8 of 8)

- [Audio Transformers](https://mlsanity.com/paper/2105.00335/) (2021)
- [A Generative Model for Raw Audio Using Transformer Architectures](https://mlsanity.com/paper/2106.16036/) (2021)
- [An Order-Complexity Aesthetic Assessment Model for Aesthetic-aware Music Recommendation](https://mlsanity.com/paper/2402.08300/) (2024)
- [Leveraging Symmetrical Convolutional Transformer Networks for Speech to Singing Voice Style Transfer](https://mlsanity.com/paper/2208.12410/) (2022)
- [Attention is All You Need? Good Embeddings with Statistics are enough:Large Scale Audio Understanding without Transformers/ Convolutions/ BERTs/ Mixers/ Attention/ RNNs or ....](https://mlsanity.com/paper/2110.03183/) (2021)
- [An Order-Complexity Model for Aesthetic Quality Assessment of Homophony Music Performance](https://mlsanity.com/paper/2304.11521/) (2023)
- [One-Shot Acoustic Matching Of Audio Signals -- Learning to Hear Music In Any Room/ Concert Hall](https://mlsanity.com/paper/2210.15750/) (2022)
- [Content Adaptive Front End For Audio Classification](https://mlsanity.com/paper/2303.10446/) (2023)

## Abstract (arXiv)

We present an end-to-end method for transforming audio from one style to another. For the case of speech, by conditioning on speaker identities, we can train a single model to transform words spoken by multiple people into multiple target voices. For the case of music, we can specify musical instruments and achieve the same result. Architecturally, our method is a fully-differentiable sequence-to-sequence model based on convolutional and hierarchical recurrent neural networks. It is designed to capture long-term acoustic dependencies, requires minimal post-processing, and produces realistic audio transforms. Ablation studies confirm that our model can separate speaker and instrument properties from acoustic content at different receptive fields. Empirically, our method achieves competitive performance on community-standard datasets.