# LUT-NN: Empower Efficient Neural Network Inference with Centroid Learning and Table Lookup

arXiv: https://arxiv.org/abs/2302.03213 · HTML: https://mlsanity.com/paper/2302.03213/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 62 references that are in the map.
- First posted on arXiv 7 Feb 2023.

## Builds on (most cited first, 30 of 32)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [Designing Network Design Spaces](https://mlsanity.com/paper/2003.13678/) (2020)
- [Once-for-All: Train One Network and Specialize it for Efficient Deployment](https://mlsanity.com/paper/1908.09791/) (2019)
- [Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition](https://mlsanity.com/paper/1804.03209/) (2018)
- [BinaryConnect: Training Deep Neural Networks with binary weights during propagations](https://mlsanity.com/paper/1511.00363/) (2015)
- [What is the State of Neural Network Pruning?](https://mlsanity.com/paper/2003.03033/) (2020)
- [DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients](https://mlsanity.com/paper/1606.06160/) (2016)
- [Age Progression/Regression by Conditional Adversarial Autoencoder](https://mlsanity.com/paper/1702.08423/) (2017)
- [Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation](https://mlsanity.com/paper/1404.0736/) (2014)
- [Speeding up Convolutional Neural Networks with Low Rank Expansions](https://mlsanity.com/paper/1405.3866/) (2014)
- [Predicting Parameters in Deep Learning](https://mlsanity.com/paper/1306.0543/) (2013)
- [Compressing Deep Convolutional Networks using Vector Quantization](https://mlsanity.com/paper/1412.6115/) (2014)
- [And the Bit Goes Down: Revisiting the Quantization of Neural Networks](https://mlsanity.com/paper/1907.05686/) (2019)
- [LogicNets: Co-Designed Neural Networks and Circuits for Extreme-Throughput Applications](https://mlsanity.com/paper/2004.03021/) (2020)
- [ANT: Exploiting Adaptive Numerical Data Type for Low-bit Deep Neural Network Quantization](https://mlsanity.com/paper/2208.14286/) (2022)
- [Differentiable Product Quantization for End-to-End Embedding Compression](https://mlsanity.com/paper/1908.09756/) (2019)
- [Ternary Neural Networks for Resource-Efficient AI Applications](https://mlsanity.com/paper/1609.00222/) (2016)
- [Discovering Low-Precision Networks Close to Full-Precision Networks for Efficient Embedded Inference](https://mlsanity.com/paper/1809.04191/) (2018)
- [Multiplying Matrices Without Multiplying](https://mlsanity.com/paper/2106.10860/) (2021)
- [Quantization based Fast Inner Product Search](https://mlsanity.com/paper/1509.01469/) (2015)
- [SUBIC: A supervised, structured binary code for image search](https://mlsanity.com/paper/1708.02932/) (2017)
- [LCNN: Lookup-based Convolutional Neural Network](https://mlsanity.com/paper/1611.06473/) (2016)
- [Euphrates: Algorithm-SoC Co-Design for Low-Power Mobile Continuous Vision](https://mlsanity.com/paper/1803.11232/) (2018)
- [Bolt: Accelerated Data Mining with Fast Vector Compression](https://mlsanity.com/paper/1706.10283/) (2017)

## Built on by (most cited first, 20 of 20)

- [GPTVQ: The Blessing of Dimensionality for LLM Quantization](https://mlsanity.com/paper/2402.15319/) (2024)
- [T-MAC: CPU Renaissance via Table Lookup for Low-Bit LLM Deployment on Edge](https://mlsanity.com/paper/2407.00088/) (2024)
- [How Far Are We From AGI: Are LLMs All We Need?](https://mlsanity.com/paper/2405.10313/) (2024)
- [Bitnet.cpp: Efficient Edge Inference for Ternary LLMs](https://mlsanity.com/paper/2502.11880/) (2025)
- [LUT-DLA: Lookup Table as Efficient Extreme Low-Bit Deep Learning Accelerator](https://mlsanity.com/paper/2501.10658/) (2025)
- [Anatomizing Deep Learning Inference in Web Browsers](https://mlsanity.com/paper/2402.05981/) (2024)
- [LUT-LLM: Efficient Large Language Model Inference with Memory-based Computations on FPGAs](https://mlsanity.com/paper/2511.06174/) (2025)
- [LL-ViT: Edge Deployable Vision Transformers with Look Up Table Neurons](https://mlsanity.com/paper/2511.00812/) (2025)
- [Stella Nera: A Differentiable Maddness-Based Hardware Accelerator for Efficient Approximate Matrix Multiplication](https://mlsanity.com/paper/2311.10207/) (2023)
- [AdaShadow: Responsive Test-time Model Adaptation in Non-stationary Mobile Environments](https://mlsanity.com/paper/2410.08256/) (2024)
- [VoLUT: Efficient Volumetric streaming enhanced by LUT-based super-resolution](https://mlsanity.com/paper/2502.12151/) (2025)
- [LCD: Advancing Extreme Low-Bit Clustering for Large Language Models via Knowledge Distillation](https://mlsanity.com/paper/2506.12038/) (2025)
- [Pushing the Envelope of LLM Inference with Ultra-Low-Bit Quantized Models](https://mlsanity.com/paper/2508.06753/) (2025)
- [LOCALUT: Harnessing Capacity-Computation Tradeoffs for LUT-Based Inference in DRAM-PIM](https://mlsanity.com/paper/2604.04523/) (2026)
- [FASQ: Flexible Accelerated Subspace Quantization for Calibration-Free LLM Compression](https://mlsanity.com/paper/2605.04084/) (2026)
- [Breaking the 1.58-bit Barrier for Ternary LLMs](https://mlsanity.com/paper/2609.16338/) (2026)
- [Mitigating scalability challenges in LUT-based neural networks via pruning optimisations](https://mlsanity.com/paper/2407.02362/) (2024)
- [Synera: Synergistic LLM Serving across Device and Cloud at Scale](https://mlsanity.com/paper/2511.07423/) (2025)
- [Optimizing Feature Extraction for On-device Model Inference with User Behavior Sequences](https://mlsanity.com/paper/2603.21508/) (2026)
- [FluxBin: Flexible LUT-based Ultra-low-bit LLM Inference by Algorithm-Kernel Synergy](https://mlsanity.com/paper/2608.15602/) (2026)

## Abstract (arXiv)

On-device Deep Neural Network (DNN) inference consumes significant computing resources and development efforts. To alleviate that, we propose LUT-NN, the first system to empower inference by table lookup, to reduce inference cost. LUT-NN learns the typical features for each operator, named centroid, and precompute the results for these centroids to save in lookup tables. During inference, the results of the closest centroids with the inputs can be read directly from the table, as the approximated outputs without computations. LUT-NN integrates two major novel techniques: (1) differentiable centroid learning through backpropagation, which adapts three levels of approximation to minimize the accuracy impact by centroids; (2) table lookup inference execution, which comprehensively considers different levels of parallelism, memory access reduction, and dedicated hardware units for optimal performance. LUT-NN is evaluated on multiple real tasks, covering image and speech recognition, and nature language processing. Compared to related work, LUT-NN improves accuracy by 66% to 92%, achieving similar level with the original models. LUT-NN reduces the cost at all dimensions, including FLOPs (≤ 16x), model size (≤ 7x), latency (≤ 6.8x), memory (≤ 6.5x), and power (≤ 41.7%).