# SLMQuant:Benchmarking Small Language Model Quantization for Practical Deployment

arXiv: https://arxiv.org/abs/2511.13023 · HTML: https://mlsanity.com/paper/2511.13023/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 25 references that are in the map.
- First posted on arXiv 17 Nov 2025.

## Builds on (most cited first, 15 of 15)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [PIQA: Reasoning about Physical Commonsense in Natural Language](https://mlsanity.com/paper/1911.11641/) (2019)
- [GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers](https://mlsanity.com/paper/2210.17323/) (2022)
- [Qwen2.5-Coder Technical Report](https://mlsanity.com/paper/2409.12186/) (2024)
- [SpinQuant: LLM quantization with learned rotations](https://mlsanity.com/paper/2405.16406/) (2024)
- [LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit](https://mlsanity.com/paper/2405.06001/) (2024)
- [PTQ4SAM: Post-Training Quantization for Segment Anything](https://mlsanity.com/paper/2405.03144/) (2024)
- [Channel Pruning Guided by Classification Loss and Feature Importance](https://mlsanity.com/paper/2003.06757/) (2020)
- [LLMCBench: Benchmarking Large Language Model Compression for Efficient Deployment](https://mlsanity.com/paper/2410.21352/) (2024)
- [PTSBench: A Comprehensive Post-Training Sparsity Benchmark Towards Algorithms and Models](https://mlsanity.com/paper/2412.07268/) (2024)
- [Model Compression using Progressive Channel Pruning](https://mlsanity.com/paper/2507.04792/) (2025)

## Built on by (most cited first, 2 of 2)

- [PocketLLM: Ultimate Compression of Large Language Models via Meta Networks](https://mlsanity.com/paper/2511.17637/) (2025)
- [Little Brains, Big Feats: Exploring Compact Language Models](https://mlsanity.com/paper/2606.30062/) (2026)

## Abstract (arXiv)

Despite the growing interest in Small Language Models (SLMs) as resource-efficient alternatives to Large Language Models (LLMs), their deployment on edge devices remains challenging due to unresolved efficiency gaps in model compression. While quantization has proven effective for LLMs, its applicability to SLMs is significantly underexplored, with critical questions about differing quantization bottlenecks and efficiency profiles. This paper introduces SLMQuant, the first systematic benchmark for evaluating LLM compression techniques when applied to SLMs. Through comprehensive multi-track evaluations across diverse architectures and tasks, we analyze how state-of-the-art quantization methods perform on SLMs. Our findings reveal fundamental disparities between SLMs and LLMs in quantization sensitivity, demonstrating that direct transfer of LLM-optimized techniques leads to suboptimal results due to SLMs' unique architectural characteristics and training dynamics. We identify key factors governing effective SLM quantization and propose actionable design principles for SLM-tailored compression. SLMQuant establishes a foundational framework for advancing efficient SLM deployment on low-end devices in edge applications, and provides critical insights for deploying lightweight language models in resource-constrained scenarios.