mlsanityData to 19 Oct 2026

Harmonic-Percussive Disentangled Neural Audio Codec for Bandwidth Extension

arXiv 2511.21580 ↗First posted 26 Nov 2025cs.SDcited by 0

On the map: Speech and audio language models → Vocoders and speech synthesis

Abstract (arXiv)

Bandwidth extension, the task of reconstructing the high-frequency components of an audio signal from its low-pass counterpart, is a long-standing problem in audio processing. While traditional approaches have evolved alongside the broader trends in signal processing, recent advances in neural architectures have significantly improved performance across a wide range of audio tasks, In this work, we extend these advances by framing bandwidth extension as an audio token prediction problem. Specifically, we train a transformer-based language model on the discrete representations produced by a disentangled neural audio codec, where the disentanglement is guided by a Harmonic-Percussive decomposition of the input signals, highlighting spectral structures particularly relevant for bandwidth extension. Our approach introduces a novel codec design that explicitly accounts for the downstream token prediction task, enabling a more effective coupling between codec structure and transformer modeling. This joint design yields high-quality reconstructions of the original signal, as measured by both objective metrics and subjective evaluations. These results highlight the importance of aligning codec disentanglement and representation learning with the generative modeling stage, and demonstrate the potential of global, representation-aware design for advancing bandwidth extension.

Key facts. No citing papers in the map yet (data through 19 Oct 2026). Placed in Vocoders and speech synthesis, one of 11 directions in Speech and audio language models. Builds on 48 of its 73 references that are in the map. First posted on arXiv 26 Nov 2025.

Builds on

48 of 73 references.

Closest ancestors · 11

Same direction, newest first

  1. 2025Soft Disentanglement in Frequency Bands for Neural Audio Codecs4 Oct
  2. 2025UniCodec: Unified Audio Codec with Single Domain-Adaptive Codebook27 Feb
  3. 2024Scaling Transformers for Low-Bitrate High-Quality Speech Coding29 Nov
  4. 2024Learning Source Disentanglement in Neural Audio Codec17 Sep
  5. 2024NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models5 Mar
  6. 2023Generalizable Zero-Shot Speaker Adaptive Speech Synthesis with Disentangled Representations24 Aug
  7. 2023HiFi-Codec: Group-residual Vector quantization for High Fidelity Audio Codec4 May
  8. 2023Speak Foreign Languages with Your Own Voice: Cross-Lingual Neural Codec Language Modeling7 Mar
  9. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers5 Jan
  10. 2022Disentangling speech from surroundings with neural embeddings29 Mar
  11. 2021SoundStream: An End-to-End Neural Audio Codec7 Jul

Other references · 37

  1. 2017Attention Is All You Needcited by 61,180
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,923
  3. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,879
  4. 2017Neural Discrete Representation Learningcited by 4,843
  5. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 4,385
  6. 2019The Curious Case of Neural Text Degenerationcited by 2,954
  7. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,404
  8. 2022High Fidelity Neural Audio Compressioncited by 917
  9. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 736
  10. 2024Moshi: a speech-text foundation model for real-time dialoguecited by 711
  11. 2018SDR - half-baked or well done?cited by 626
  12. 2023High-Fidelity Audio Compression with Improved RVQGANcited by 619
25 more
  1. 2023Simple and Controllable Music Generationcited by 540
  2. 2021Speech Resynthesis from Discrete Disentangled Self-Supervised Representationscited by 287
  3. 2022Masked Autoencoders that Listencited by 286
  4. 2020DDSP: Differentiable Digital Signal Processingcited by 190
  5. 2020ViSQOL v3: An Open Source Production Ready Objective Speech and Audio Metriccited by 120
  6. 2021Generating Diverse Structure for Image Inpainting With Hierarchical VQ-VAEcited by 107
  7. 2024SemantiCodec: An Ultra Low Bitrate Semantic Audio Codec for General Soundcited by 84
  8. 2022SQ-VAE: Variational Bayes on Discrete Representation with Self-annealed Stochastic Quantizationcited by 75
  9. 2021NU-Wave: A Diffusion Probabilistic Model for Neural Audio Upsamplingcited by 52
  10. 2018The 2018 Signal Separation Evaluation Campaigncited by 45
  11. 2023AudioSR: Versatile Audio Super-resolution at Scalecited by 45
  12. 2023Disentanglement via Latent Quantizationcited by 43
  13. 2022NU-Wave 2: A General Neural Audio Upsampling Model for Various Sampling Ratescited by 39
  14. 2024MaskSR: Masked Language Model for Full-band Speech Restorationcited by 37
  15. 2017Audio Super Resolution using Neural Networkscited by 28
  16. 2022BEHM-GAN: Bandwidth Extension of Historical Music using Generative Adversarial Networkscited by 15
  17. 2021Hierarchical disentangled representation learning for singing voice conversioncited by 13
  18. 2025A2SB: Audio-to-Audio Schrodinger Bridgescited by 12
  19. 2024Gull: A Generative Multifunctional Audio Codeccited by 10
  20. 2024Apollo: Band-sequence Modeling for High-Quality Audio Restorationcited by 9
  21. 2025SpectroStream: A Versatile Neural Codec for General Audiocited by 7
  22. 2024Vector Quantized Diffusion Model Based Speech Bandwidth Extensioncited by 6
  23. 2020OrchideaSOL: a dataset of extended instrumental techniques for computer-aided orchestrationcited by 5
  24. 2025UniverSR: Unified and Versatile Audio Super-Resolution via Vocoder-Free Flow Matchingcited by 4
  25. 2023Efficient bandwidth extension of musical signals using a differentiable harmonic plus noise modelcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025D'esentrelacement Fr'equentiel Doux pour les Codecs Audio Neuronaux
  2. 2025BSCodec: A Band-Split Neural Codec for High-Quality Universal Audio Reconstruction
  3. 2025DeCodec: Rethinking Audio Codecs as Universal Disentangled Representation Learners
  4. 2026HARP: Harmonic-Aware Residual Partitioning for Neural Audio Codecs
  5. 2024A Closer Look at Neural Codec Resynthesis: Bridging the Gap between Codec and Waveform Generation
  6. 2024HILCodec: High-Fidelity and Lightweight Neural Audio Codec
  7. 2026On the Geometry of Music Bandwidth Extension in Latent Spaces of Audio Codecs
  8. 2026CodecFlow: Efficient Bandwidth Extension via Conditional Flow Matching in Neural Codec Latent Space