A Framework for Robust Lossy Compression of Heavy-Tailed Sources
Not on the map. Why?
Abstract (arXiv)
We study the rate-distortion problem for both scalar and vector memoryless heavy-tailed α-stable sources (0 < α < 2). Using a recently defined notion of ``strength" as a power measure, we derive the rate-distortion function for α-stable sources subject to a constraint on the strength of the error and show it to be logarithmic in the strength-to-distortion ratio. We show how our framework paves the way for finding optimal quantizers for α-stable sources and other general heavy-tailed ones. In addition, we study high-rate scalar quantizers and show that uniform ones are asymptotically optimal under the error-strength distortion measure. We compare uniform Gaussian and Cauchy quantizers and show that more representation points for the Cauchy source are required to guarantee the same quantization quality. Our findings generalize the well-known results of rate-distortion and quantization of Gaussian sources (α = 2) under a quadratic distortion measure.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 4 of its 39 references that are in the map. First posted on arXiv 13 Nov 2024.
Builds on
4 of 39 references.
References · 4
- 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
- 2014Compressing Deep Convolutional Networks using Vector Quantizationcited by 195
- 2019Low-bit Quantization of Neural Networks for Efficient Inferencecited by 193
- 2018Rate Distortion For Model Compression: From Theory To Practicecited by 15
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Lessons Learned on the Path to Guaranteeing the Error Bound in Lossy Quantizers
- 2022Quantizing Heavy-tailed Data in Statistical Estimation: (Near) Minimax Rates, Covariate Quantization, and Uniform Recovery
- 2025TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate
- 2022High Dimensional Statistical Estimation under Uniformly Dithered One-bit Quantization
- 2021Rate distortion comparison of a few gradient quantizers
- 2026Provable Quantization with Randomized Hadamard Transform
- 2024Improved Quantization Strategies for Managing Heavy-tailed Gradients in Distributed Learning
- 2026FidelityQ: Preserving Predictive Fidelity in Ultra-Low-Bit Language Model Quantization