# Pyramid Vector Quantization for Deep Learning

arXiv: https://arxiv.org/abs/1704.02681 · HTML: https://mlsanity.com/paper/1704.02681/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Apr 2017.

## Built on by (most cited first, 1 of 1)

- [Pyramid Vector Quantization for LLMs](https://mlsanity.com/paper/2410.16926/) (2024)

## Abstract (arXiv)

This paper explores the use of Pyramid Vector Quantization (PVQ) to reduce the computational cost for a variety of neural networks (NNs) while, at the same time, compressing the weights that describe them. This is based on the fact that the dot product between an N dimensional vector of real numbers and an N dimensional PVQ vector can be calculated with only additions and subtractions and one multiplication. This is advantageous since tensor products, commonly used in NNs, can be re-conduced to a dot product or a set of dot products. Finally, it is stressed that any NN architecture that is based on an operation that can be re-conduced to a dot product can benefit from the techniques described here.