# A Framework for Robust Lossy Compression of Heavy-Tailed Sources

arXiv: https://arxiv.org/abs/2411.08549 · HTML: https://mlsanity.com/paper/2411.08549/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 4 of its 39 references that are in the map.
- First posted on arXiv 13 Nov 2024.

## Builds on (most cited first, 4 of 4)

- [SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models](https://mlsanity.com/paper/2211.10438/) (2022)
- [Compressing Deep Convolutional Networks using Vector Quantization](https://mlsanity.com/paper/1412.6115/) (2014)
- [Low-bit Quantization of Neural Networks for Efficient Inference](https://mlsanity.com/paper/1902.06822/) (2019)
- [Rate Distortion For Model Compression: From Theory To Practice](https://mlsanity.com/paper/1810.06401/) (2018)

## Abstract (arXiv)

We study the rate-distortion problem for both scalar and vector memoryless heavy-tailed α-stable sources (0 < α < 2). Using a recently defined notion of ``strength" as a power measure, we derive the rate-distortion function for α-stable sources subject to a constraint on the strength of the error and show it to be logarithmic in the strength-to-distortion ratio. We show how our framework paves the way for finding optimal quantizers for α-stable sources and other general heavy-tailed ones. In addition, we study high-rate scalar quantizers and show that uniform ones are asymptotically optimal under the error-strength distortion measure. We compare uniform Gaussian and Cauchy quantizers and show that more representation points for the Cauchy source are required to guarantee the same quantization quality. Our findings generalize the well-known results of rate-distortion and quantization of Gaussian sources (α = 2) under a quadratic distortion measure.