# Latent Granular Resynthesis using Neural Audio Codecs

arXiv: https://arxiv.org/abs/2507.19202 · HTML: https://mlsanity.com/paper/2507.19202/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 15 references that are in the map.
- First posted on arXiv 25 Jul 2025.

## Builds on (most cited first, 13 of 13)

- [SoundStream: An End-to-End Neural Audio Codec](https://mlsanity.com/paper/2107.03312/) (2021)
- [High Fidelity Neural Audio Compression](https://mlsanity.com/paper/2210.13438/) (2022)
- [High-Fidelity Audio Compression with Improved RVQGAN](https://mlsanity.com/paper/2306.06546/) (2023)
- [Fast Timing-Conditioned Latent Audio Diffusion](https://mlsanity.com/paper/2402.04825/) (2024)
- [Long-form music generation with latent diffusion](https://mlsanity.com/paper/2404.10301/) (2024)
- [RAVE: A variational autoencoder for fast and high-quality neural audio synthesis](https://mlsanity.com/paper/2111.05011/) (2021)
- [Diff-A-Riff: Musical Accompaniment Co-creation via Latent Diffusion Models](https://mlsanity.com/paper/2406.08384/) (2024)
- [Music2Latent: Consistency Autoencoders for Latent Audio Compression](https://mlsanity.com/paper/2408.06500/) (2024)
- [A Universal Music Translation Network](https://mlsanity.com/paper/1805.07848/) (2018)
- [Improving Musical Accompaniment Co-creation via Diffusion Transformers](https://mlsanity.com/paper/2410.23005/) (2024)
- [Neural Granular Sound Synthesis](https://mlsanity.com/paper/2008.01393/) (2020)
- [Music2Latent2: Audio Compression with Summary Embeddings and Autoregressive Decoding](https://mlsanity.com/paper/2501.17578/) (2025)
- [The Concatenator: A Bayesian Approach To Real Time Concatenative Musaicing](https://mlsanity.com/paper/2411.04366/) (2024)

## Built on by (most cited first, 2 of 2)

- [Latent-Mark: An Audio Watermark Robust to Neural Codec Compression](https://mlsanity.com/paper/2603.05310/) (2026)
- [Mix2Morph: Learning Sound Morphing from Noisy Mixes](https://mlsanity.com/paper/2601.20426/) (2026)

## Abstract (arXiv)

We introduce a novel technique for creative audio resynthesis that operates by reworking the concept of granular synthesis at the latent vector level. Our approach creates a "granular codebook" by encoding a source audio corpus into latent vector segments, then matches each latent grain of a target audio signal to its closest counterpart in the codebook. The resulting hybrid sequence is decoded to produce audio that preserves the target's temporal structure while adopting the source's timbral characteristics. This technique requires no model training, works with diverse audio materials, and naturally avoids the discontinuities typical of traditional concatenative synthesis through the codec's implicit interpolation during decoding. We include supplementary material at https://github.com/naotokui/latentgranular/ , as well as a proof-of-concept implementation to allow users to experiment with their own sounds at https://huggingface.co/spaces/naotokui/latentgranular .