mlsanityData to 14 Oct 2026

FlexQuant: Elastic Quantization Framework for Locally Hosted LLM on Edge Devices

arXiv 2501.07139 ↗First posted 13 Jan 2025cs.AI cs.PFcited by 4

Not on the map. Why?

Abstract (arXiv)

Deploying LLMs on edge devices presents serious technical challenges. Memory elasticity is crucial for edge devices with unified memory, where memory is shared and fluctuates dynamically. Existing solutions suffer from either poor transition granularity or high storage costs. We propose FlexQuant, a novel elasticity framework that generates an ensemble of quantized models, providing an elastic hosting solution with 31x more deployment options, 15x granularity improvement, and 10x storage reduction compared to SoTA methods. FlexQuant works with most quantization methods and creates a family of trade-off options under various storage limits through our pruning method. It brings great performance and flexibility to the edge deployment of LLMs.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 20 of its 35 references that are in the map. First posted on arXiv 13 Jan 2025.

Builds on

20 of 35 references.

References · 20

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  6. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  7. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  8. 2016Pointer Sentinel Mixture Modelscited by 3,096
  9. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  10. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  11. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  12. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
8 more
  1. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 621
  2. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 526
  3. 2021A White Paper on Neural Network Quantizationcited by 472
  4. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 367
  5. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 294
  6. 2023LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Modelscited by 196
  7. 2024Any-Precision LLM: Low-Cost Deployment of Multiple, Different-Sized LLMscited by 54
  8. 2024Elastic On-Device LLM Servicecited by 3

Built on by

Cited by year

Citing papers per year: 2025: 4.

4 papers.

  1. 2025Sustainable LLM Inference for Edge AI: Evaluating Quantized LLMs for Energy Efficiency, Output Accuracy, and Inference Latencycited by 34
  2. 2025Debt Collection Negotiations with Large Language Models: An Evaluation System and Optimizing Decision Making with Multi-Agentcited by 10
  3. 2025LowRA: Accurate and Efficient LoRA Fine-Tuning of LLMs under 2 Bitscited by 6
  4. 2025DisCEdge: Distributed Context Management for Large Language Models at the Edgecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Systematic Evaluation of On-Device LLMs: Quantization, Performance, and Resources
  2. 2026TOM: A Ternary Read-only Memory Accelerator for LLM-powered Edge Intelligence
  3. 2026APreQEL: Adaptive Mixed Precision Quantization For Edge LLMs
  4. 2025ELUTQ: Optimizing Quantization Accuracy under LUT-Based Computation for Edge LLMs
  5. 2025UniQL: Unified Quantization and Low-rank Compression for Adaptive Edge LLMs
  6. 2026PALUTE: Processing-In-Memory Acceleration via Lookup Table for Edge LLM Inference
  7. 2026QMC: Efficient SLM Edge Inference via Outlier-Aware Quantization and Emergent Memories Co-Design
  8. 2025DILEMMA: Joint LLM Quantization and Distributed LLM Inference Over Edge Computing Systems