OneComp: One-Line Revolution for Generative AI Model Compression
Not on the map. Why?
Abstract (arXiv)
Deploying foundation models is increasingly constrained by memory footprint, latency, and hardware costs. Post-training compression can mitigate these bottlenecks by reducing the precision of model parameters without significantly degrading performance; however, its practical implementation remains challenging as practitioners navigate a fragmented landscape of quantization algorithms, precision budgets, data-driven calibration strategies, and hardware-dependent execution regimes. We present OneComp, an open-source compression framework that transforms this expert workflow into a reproducible, resource-adaptive pipeline. Given a model identifier and available hardware, OneComp automatically inspects the model, plans mixed-precision assignments, and executes progressive quantization stages, ranging from layer-wise compression to block-wise refinement and global refinement. A key architectural choice is treating the first quantized checkpoint as a deployable pivot, ensuring that each subsequent stage improves the same model and that quality increases as more compute is invested. By converting state-of-the-art compression research into an extensible, open-source, hardware-aware pipeline, OneComp bridges the gap between algorithmic innovation and production-grade model deployment.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 49 of its 59 references that are in the map. First posted on arXiv 30 Mar 2026.
Builds on
49 of 59 references.
References · 49
- 2023GPT-4 Technical Reportcited by 18,132
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
- 2025Qwen3 Technical Reportcited by 8,195
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
- 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
- 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
- 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
- 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
- 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
- 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
37 more
- 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
- 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
- 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 598
- 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 571
- 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 557
- 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 517
- 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 464
- 2021A White Paper on Neural Network Quantizationcited by 453
- 2024SpinQuant: LLM quantization with learned rotationscited by 446
- 2016Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activationscited by 413
- 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 403
- 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 397
- 2019AutoML: A Survey of the State-of-the-Artcited by 388
- 2015Deep Learning with Limited Numerical Precisioncited by 368
- 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 354
- 2023LLM-QAT: Data-Free Quantization Aware Training for Large Language Modelscited by 320
- 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 285
- 2022The case for 4-bit precision: k-bit Inference Scaling Lawscited by 247
- 2024Extreme Compression of Large Language Models via Additive Quantizationcited by 223
- 2023BitNet: Scaling 1-bit Transformers for Large Language Modelscited by 175
- 2024LLM Inference Unveiled: Survey and Roofline Model Insightscited by 147
- 2020Efficient Riemannian Optimization on the Stiefel Manifold via the Cayley Transformcited by 79
- 2024OneBit: Towards Extremely Low-bit Large Language Modelscited by 77
- 2024PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compressioncited by 59
- 2024VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Modelscited by 53
- 2023On the Impact of Calibration Data in Post-training Quantization and Pruningcited by 51
- 2025Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantizationcited by 36
- 2023CBQ: Cross-Block Quantization for Large Language Modelscited by 34
- 2024APTQ: Attention-aware Post-Training Mixed-Precision Quantization for Large Language Modelscited by 33
- 2024A Survey of Low-bit Large Language Models: Basics, Systems, and Algorithmscited by 30
- 2025Benchmarking Post-Training Quantization in LLMs: Comprehensive Taxonomy, Unified Evaluation, and Comparative Analysiscited by 12
- 2025LittleBit: Ultra Low-Bit Quantization via Latent Factorizationcited by 11
- 2025LPCD: Unified Framework from Layer-Wise to Submodule Quantizationcited by 6
- 2020Technical Report: NEMO DNN Quantization for Deployment Modelcited by 5
- 2025LoaQ: Layer-wise Output Approximation Quantizationcited by 5
- 2025Addition is almost all you need: Compressing large language models with double binary factorizationcited by 3
- 2025More Than Bits: Multi-Envelope Double Binary Factorization for Extreme Quantizationcited by 3
Built on by
Citing papers per year: 2026: 3.
3 papers.