# Audio Decoding by Inverse Problem Solving

arXiv: https://arxiv.org/abs/2409.07858 · HTML: https://mlsanity.com/paper/2409.07858/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 31 references that are in the map.
- First posted on arXiv 12 Sep 2024.

## Builds on (most cited first, 19 of 19)

- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Improved Techniques for Training Score-Based Generative Models](https://mlsanity.com/paper/2006.09011/) (2020)
- [Multi-Source Diffusion Models for Simultaneous Music Generation and Separation](https://mlsanity.com/paper/2302.02257/) (2023)
- [Solving Audio Inverse Problems with a Diffusion Model](https://mlsanity.com/paper/2210.15228/) (2022)
- [JPEG Artifact Correction using Denoising Diffusion Restoration Models](https://mlsanity.com/paper/2209.11888/) (2022)
- [Wavenet based low rate speech coding](https://mlsanity.com/paper/1712.01120/) (2017)
- [Full-band General Audio Synthesis with Score-based Diffusion](https://mlsanity.com/paper/2210.14661/) (2022)
- [Diffusion Models for Audio Restoration](https://mlsanity.com/paper/2402.09821/) (2024)
- [Blind Audio Bandwidth Extension: A Diffusion-Based Zero-Shot Approach](https://mlsanity.com/paper/2306.01433/) (2023)
- [Parallel and Flexible Sampling from Autoregressive Models via Langevin Dynamics](https://mlsanity.com/paper/2105.08164/) (2021)
- [High-quality speech coding with SampleRNN](https://mlsanity.com/paper/1811.03021/) (2018)
- [A Diffusion-Based Generative Equalizer for Music Restoration](https://mlsanity.com/paper/2403.18636/) (2024)
- [Consistency Model is an Effective Posterior Sample Approximation for Diffusion Inverse Solvers](https://mlsanity.com/paper/2403.12063/) (2024)
- [Source coding of audio signals with a generative model](https://mlsanity.com/paper/2001.09847/) (2020)
- [On tuning consistent annealed sampling for denoising score matching](https://mlsanity.com/paper/2104.03725/) (2021)
- [High Quality Audio Coding with MDCTNet](https://mlsanity.com/paper/2212.04583/) (2022)
- [ODAQ: Open Dataset of Audio Quality](https://mlsanity.com/paper/2401.00197/) (2023)
- [Distribution Preserving Source Separation With Time Frequency Predictive Models](https://mlsanity.com/paper/2303.05896/) (2023)

## Built on by (most cited first, 1 of 1)

- [Compression of Higher Order Ambisonics with Multichannel RVQGAN](https://mlsanity.com/paper/2411.12008/) (2024)

## Abstract (arXiv)

We consider audio decoding as an inverse problem and solve it through diffusion posterior sampling. Explicit conditioning functions are developed for input signal measurements provided by an example of a transform domain perceptual audio codec. Viability is demonstrated by evaluating arbitrary pairings of a set of bitrates and task-agnostic prior models. For instance, we observe significant improvements on piano while maintaining speech performance when a speech model is replaced by a joint model trained on both speech and piano. With a more general music model, improved decoding compared to legacy methods is obtained for a broad range of content types and bitrates. The noisy mean model, underlying the proposed derivation of conditioning, enables a significant reduction of gradient evaluations for diffusion posterior sampling, compared to methods based on Tweedie's mean. Combining Tweedie's mean with our conditioning functions improves the objective performance. An audio demo is available at https://dpscodec-demo.github.io/.