mlsanityData to 14 Oct 2026

Gull: A Generative Multifunctional Audio Codec

arXiv 2404.04947 ↗First posted 7 Apr 2024eess.AS cs.AI cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

We introduce Gull, a generative multifunctional audio codec. Gull is a general purpose neural audio compression and decompression model which can be applied to a wide range of tasks and applications such as real-time communication, audio super-resolution, and codec language models. The key components of Gull include (1) universal-sample-rate modeling via subband modeling schemes motivated by recent progress in audio source separation, (2) gain-shape representations motivated by traditional audio codecs, (3) improved residual vector quantization modules, (4) elastic decoder network that enables user-defined model size and complexity during inference time, (5) built-in ability for audio super-resolution without the increase of bitrate. We compare Gull with existing traditional and neural audio codecs and show that Gull is able to achieve on par or better performance across various sample rates, bitrates and model complexities in both subjective and objective evaluation metrics.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 38 of its 61 references that are in the map. First posted on arXiv 7 Apr 2024.

Builds on

38 of 61 references.

References · 38

  1. 2016Layer Normalizationcited by 5,669
  2. 2022Generative Adversarial Networkscited by 5,627
  3. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,636
  4. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,387
  5. 2016Least Squares Generative Adversarial Networkscited by 1,001
  6. 2021SoundStream: An End-to-End Neural Audio Codeccited by 771
  7. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 752
  8. 2022Autoregressive Image Generation using Residual Quantizationcited by 722
  9. 2016Language Modeling with Gated Convolutional Networkscited by 702
  10. 2022High Fidelity Neural Audio Compressioncited by 687
  11. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 616
  12. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 596
26 more
  1. 2023Simple and Controllable Music Generationcited by 488
  2. 2023MusicLM: Generating Music From Textcited by 448
  3. 2023High-Fidelity Audio Compression with Improved RVQGANcited by 431
  4. 2018Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separationcited by 406
  5. 2023NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizerscited by 235
  6. 2019Rethinking Lossy Compression: The Rate-Distortion-Perception Tradeoffcited by 211
  7. 2022NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Qualitycited by 152
  8. 2023EXPRESSO: A Benchmark and Analysis of Discrete Expressive Speech Resynthesiscited by 98
  9. 2020ViSQOL v3: An Open Source Production Ready Objective Speech and Audio Metriccited by 79
  10. 2022Music Source Separation with Band-split RNNcited by 70
  11. 2023SpeechX: Neural Codec Language Model as a Versatile Speech Transformercited by 64
  12. 2022ICASSP 2022 Deep Noise Suppression Challengecited by 60
  13. 2019Low Bit-Rate Speech Coding with VQ-VAE and a WaveNet Decodercited by 49
  14. 2017Wavenet based low rate speech codingcited by 33
  15. 2019End-to-end Microphone Permutation and Number Invariant Multi-channel Speech Separationcited by 31
  16. 2017End-to-End Optimized Speech Coding with Deep Neural Networkscited by 22
  17. 2020Real-time Speech Frequency Bandwidth Extensioncited by 22
  18. 2021Generative Speech Coding with Predictive Variance Regularizationcited by 18
  19. 2018High-quality speech coding with SampleRNNcited by 14
  20. 2020Audio Codec Enhancement with Generative Adversarial Networkscited by 12
  21. 2023LMCodec: A Low Bitrate Speech Codec With Causal Transformer Modelscited by 9
  22. 2023Subnetwork-to-go: Elastic Neural Network with Dynamic Training and Customizable Inferencecited by 9
  23. 2021HARP-Net: Hyper-Autoencoded Reconstruction Propagation for Scalable Neural Audio Codingcited by 7
  24. 2020Source-Aware Neural Speech Coding for Noisy Speech Compressioncited by 5
  25. 2021Scalable and Efficient Neural Speech Coding: A Hybrid Designcited by 5
  26. 2016A High-Quality Speech and Audio Codec With Less Than 10 ms Delaycited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 1; 2026: 1.

5 papers.

  1. 2025NanoCodec: Towards High-Quality Ultra Fast Speech LLM Inferencecited by 15
  2. 2024Low Frame-rate Speech Codec: a Codec Designed for Fast High-quality Speech LLM Training and Inferencecited by 11
  3. 2024Apollo: Band-sequence Modeling for High-Quality Audio Restorationcited by 8
  4. 2024Learning Source Disentanglement in Neural Audio Codeccited by 6
  5. 2026Stable Autoregressive Speech Generation with Low-Frame-Rate High-Dimensional Continuous Tokenscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026GS-Codec: A Gaussian-Splatting Bottleneck for Neural Audio Coding
  2. 2025FlowDec: A flow-based full-band general audio codec with high perceptual quality
  3. 2026UniSRCodec: Unified and Low-Bitrate Single Codebook Codec with Sub-Band Reconstruction
  4. 2025QINCODEC: Neural Audio Compression with Implicit Neural Codebooks
  5. 2024SemantiCodec: An Ultra Low Bitrate Semantic Audio Codec for General Sound
  6. 2025Finite Scalar Quantization Enables Redundant and Transmission-Robust Neural Audio Compression at Low Bit-rates
  7. 2026Switchcodec: Adaptive residual-expert sparse quantization for high-fidelity neural audio coding
  8. 2025CoDiCodec: Unifying Continuous and Discrete Compressed Representations of Audio