mlsanityData to 14 Oct 2026

Quant-Trim in Practice: Improved Cross-Platform Low-Bit Deployment on Edge NPUs

arXiv 2511.15300 ↗First posted 19 Nov 2025cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

Specialized edge accelerators rely on low-bit quantization, but vendor compilers differ in scaling, clipping, and kernel support, often as black boxes. The same floating-point (FP) checkpoint can therefore yield inconsistent accuracy across backends, forcing practitioners to tweak flags or refactor models to vendor-friendly operator subsets. We introduce Quant-Trim, a training-phase method that produces a hardware-neutral checkpoint robust to backend and precision choices. It combines progressive fake quantization to align training with the deployed integer grid and reverse pruning to tame outlier-driven scale inflation while preserving learnability. Quant-Trim is agnostic to quantization schemes (symmetric/asymmetric, per-tensor/per-channel, INT8/INT4) and requires no vendor-specific graph changes. Across models and tasks, it narrows the FP-to-low-bit gap, reduces dependence on compiler heuristics/calibration, and avoids per-backend retraining. We report accuracy and edge metrics latency, throughput, energy per inference, and cost under static/dynamic activation scaling and varying operator coverage.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 31 of its 34 references that are in the map. First posted on arXiv 19 Nov 2025.

Builds on

31 of 34 references.

References · 31

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  6. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  7. 2020End-to-End Object Detection with Transformerscited by 7,414
  8. 2019Searching for MobileNetV3cited by 1,994
  9. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  10. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  11. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  12. 2017Mixed Precision Trainingcited by 1,156
19 more
  1. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  2. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  3. 2019Learned Step Size Quantizationcited by 628
  4. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 621
  5. 2018Quantizing deep convolutional networks for efficient inference: A whitepapercited by 521
  6. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 479
  7. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 467
  8. 2016Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activationscited by 426
  9. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 365
  10. 2019A Study of BFLOAT16 for Deep Learning Trainingcited by 232
  11. 2019Understanding Straight-Through Estimator in Training Activation Quantized Neural Netscited by 231
  12. 2022FP8 Formats for Deep Learningcited by 196
  13. 2024SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Modelscited by 170
  14. 2024DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMscited by 167
  15. 2023OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Modelscited by 114
  16. 2021Scalable Marginal Likelihood Estimation for Model Selection in Deep Learningcited by 106
  17. 2013Estimating or Propagating Gradients Through Stochastic Neuronscited by 55
  18. 2024Shaving Weights with Occam's Razor: Bayesian Sparsification for Neural Networks Using the Marginal Likelihoodcited by 7
  19. 2024Rotated Runtime Smooth: Training-Free Activation Smoother for accurate INT4 inferencecited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Quantizing Recursive Reasoning Modelscited by 2
  2. 2026Is INT8 Portable? A Cross-Platform Measurement Study of Quantized Inference on Embedded and Automotive Acceleratorscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021VS-Quant: Per-vector Scaled Quantization for Accurate Low-Precision Neural Network Inference
  2. 2025TeLLMe: An Energy-Efficient Ternary LLM Accelerator for Prefilling and Decoding on Edge FPGAs
  3. 2023Agile-Quant: Activation-Guided Quantization for Faster Inference of LLMs on the Edge
  4. 2026Bit-by-Bit: Progressive QAT Strategy with Outlier Channel Splitting for Stable Low-Bit LLMs
  5. 2026StableQAT: Stable Quantization-Aware Training at Ultra-Low Bitwidths
  6. 2022Low- and Mixed-Precision Inference Accelerators
  7. 2025TeLLMe v2: An Efficient End-to-End Ternary LLM Prefill and Decode Accelerator with Table-Lookup Matmul on Edge FPGAs
  8. 2025HALO: Hardware-aware quantization with low critical-path-delay weights for LLM acceleration