mlsanityData to 14 Oct 2026

Post-Training Ternarization of Qwen3-4B Capability, Effective Bit Budget, Storage Compression, and Deployment

arXiv 2609.01962 ↗First posted 2 Sep 2026cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Ultra-low-bit language models can reduce storage and memory bandwidth, but a nominal "1.58-bit" label does not fully describe the stored representation, retained capability, or runtime behavior. We study an end-to-end post-training conversion of Qwen, an instruction-tuned 4B-parameter model, using KOTMS rotation, E2M-ATQ ternarization, and GPTQ-style error compensation from TWLA. The experiment is weight-only: activations remain at 16-bit precision, so ILA-AMP is omitted. We evaluate effective bit accounting, task capability retention, perplexity, calibration sensitivity, checkpoint composition, and deployment behavior. The final conversion uses 1.641 effective bits per weight for quantized linear weights, with 81.62% of model parameters targeted. Across ten scored capability comparisons, accuracy falls from 64.5% to 54.7%. Degradation is uneven: BoolQ retains 84.6% chance-corrected teacher performance, while ARC-Challenge retains 43.8%. Perplexity rises from 13.639 to 18.748 on WikiText-2, 24.700 to 31.992 on PTB, and 19.831 to 28.966 on C4. A subsequent packing run preserves the ternary planes and scales, reducing reported model size from 8.29 GiB to 3.96 GiB with essentially unchanged perplexity. A separate third-party packing attempt was lossy and is excluded from the primary artifact claim. The packed artifact has not been benchmarked end-to-end for task accuracy or generation throughput. A preliminary Triton GEMV microbenchmark is 4.6x slower than FP16 cuBLAS on one tested shape. We therefore do not claim that compression alone yields faster inference.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 11 of its 24 references that are in the map. First posted on arXiv 2 Sep 2026.

Builds on

11 of 24 references.

References · 11

  1. 2017Attention Is All You Needcited by 60,361
  2. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  3. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  4. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  5. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 424
  6. 2024The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bitscited by 308
  7. 2025PTQTP: Post-Training Quantization to Trit-Planes for Large Language Modelscited by 9
  8. 2026TWLA: Achieving Ternary Weights and Low-Bit Activations for LLMs via Post-Training Quantizationcited by 8
  9. 2026CAT-Q: Cost-efficient and Accurate Ternary Quantization for LLMscited by 6
  10. 2026Attend to Your Own Thoughts: Breaking the Barrier for Post-Training Quantization of Reasoning LLMs through the Lens of 1.58-Bit Quantizationcited by 5
  11. 2026Capability-Stratified Degradation in Ternary Language Modelscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Scaling Post-Training Ternarisation to Qwen3-8B Capability Retention, Reproduction, Lossless Packing, and Packed Execution
  2. 2026Same Bit Width, Different Outcomes: Post-Training Quantization of Text-to-Speech Across Architectures
  3. 2025An Empirical Study of Qwen3 Quantization
  4. 2025Scaling Law for Quantization-Aware Training
  5. 2026Quantization-Aware Healing: A Practical Recipe for Recovering Compressed, 4-Bit LLMs
  6. 2026Quantization Effects on Tool-Failure Recovery Vary Across Prompts and Evaluation Designs
  7. 2025Continual Quantization-Aware Pre-Training: When to transition from 16-bit to 1.58-bit pre-training for BitNet language models?
  8. 2026Mapping the Schedule x Bit-Width Boundary in Sub-100M Quantisation-Aware Training