mlsanityData to 14 Oct 2026

Optimal and Near-Optimal Adaptive Vector Quantization

arXiv 2402.03158 ↗First posted 5 Feb 2024cs.LG cs.DS cs.ITcited by 2

Not on the map. Why?

Abstract (arXiv)

Quantization is a fundamental optimization for many machine-learning use cases, including compressing gradients, model weights and activations, and datasets. The most accurate form of quantization is adaptive, where the error is minimized with respect to a given input, rather than optimizing for the worst case. However, optimal adaptive quantization methods are considered infeasible in terms of both their runtime and memory requirements. We revisit the Adaptive Vector Quantization (AVQ) problem and present algorithms that find optimal solutions with asymptotically improved time and space complexity. We also present an even faster near-optimal algorithm for large inputs. Our experiments show our algorithms may open the door to using AVQ more extensively in a variety of machine learning applications.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 22 of its 40 references that are in the map. First posted on arXiv 5 Feb 2024.

Builds on

22 of 40 references.

References · 22

  1. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
  2. 2017Mixed Precision Trainingcited by 1,156
  3. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 621
  4. 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 575
  5. 2019Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processingcited by 421
  6. 2018Sparsified SGD with Memorycited by 376
  7. 2019Q-BERT: Hessian Based Ultra Low Precision Quantization of BERTcited by 359
  8. 2019PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimizationcited by 238
  9. 2018Expanding the Reach of Federated Learning by Reducing Client Resource Requirementscited by 163
  10. 2016Distributed Mean Estimation with Limited Communicationcited by 136
  11. 2019Natural Compression for Distributed Deep Learningcited by 90
  12. 2021EDEN: Communication-Efficient and Robust Distributed Mean Estimation for Federated Learningcited by 55
10 more
  1. 2021DRIVE: One-bit Distributed Mean Estimationcited by 52
  2. 2023Dataset Quantizationcited by 48
  3. 2020Uncertainty Principle for Communication Compression in Distributed and Federated Learning and the Search for an Optimal Compressorcited by 45
  4. 2020Adaptive Gradient Quantization for Data-Parallel SGDcited by 45
  5. 2016Randomized Distributed Mean Estimation: Accuracy vs Communicationcited by 39
  6. 2023DoCoFL: Downlink Compression for Cross-Device Federated Learningcited by 28
  7. 2019NUQSGD: Provably Communication-efficient Data-parallel SGD via Nonuniform Quantizationcited by 18
  8. 2023THC: Accelerating Distributed Deep Learning Using Tensor Homomorphic Compressioncited by 15
  9. 2020How to send a real number using a single bit (and some shared randomness)cited by 12
  10. 2022QUIC-FL: Quick Unbiased Compression for Federated Learningcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025PacTrain: Pruning and Adaptive Sparse Gradient Compression for Efficient Collective Communication in Distributed Deep Learningcited by 0
  2. 2026VQ-bench: A Composable Vector Quantization Frameworkcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Inner Product Aware Quantization: Provably Fast, Accurate, and Adaptive Algorithms
  2. 2026Entropy-Constrained Adaptive Stochastic Quantization
  3. 2024Preventing Local Pitfalls in Vector Quantization via Optimal Transport
  4. 2023Soft Convex Quantization: Revisiting Vector Quantization with Convex Optimization
  5. 2024Addressing Representation Collapse in Vector Quantized Models with One Linear Layer
  6. 2025Enhancing Vector Quantization with Distributional Matching: A Theoretical and Empirical Study
  7. 2026Beyond Stationarity: Rethinking Codebook Collapse in Vector Quantization
  8. 2026RSLM: Training-Free Vector Quantization for Approximate Nearest Neighbor Search