mlsanityData to 7 Oct 2026

Scaling Post-Training Ternarisation to Qwen3-8B Capability Retention, Reproduction, Lossless Packing, and Packed Execution

arXiv 2609.09240 ↗First posted 8 Sep 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Ultra-low-bit language models promise reductions in storage and memory traffic, but a nominal "1.58-bit" label does not specify the deployed representation or its execution cost. We study a scale-up of an aggressive post-training conversion pipeline from Qwen3-4B to Qwen3-8B. The conversion uses KOTMS rotation, E2M-ATQ adaptive ternarisation, and GPTQ-style error compensation in a weight-only A16 configuration. We do not claim these algorithms as new. Our contribution is the end-to-end scale-up characterisation: an external reproduction gate, matched 4B/8B capability analysis, cross-corpus perplexity, effective-bit accounting, lossless lattice-aware packing, and direct packed execution. The 8B model reaches a three-corpus perplexity ratio of 1.361x, with WikiText-2, C4, and PTB ratios of 1.318x, 1.393x, and 1.371x. On eight zero-shot tasks at n = 500, mean accuracy is 64.6% versus 72.4% for FP16, corresponding to 78.5% chance-corrected retention and a 7.8-point absolute cost. The matched 4B run retains 69.6%, yielding an 8.9-point 8B advantage. The packed checkpoint is 8.24 GiB and preserves the recorded perplexity to measurement precision. Direct packed execution reaches 15.52 tokens/s in 7.35 GiB, while a preliminary packed GEMV remains slower than FP16 cuBLAS. The result is a validated scale-up baseline: model size improves robustness to aggressive post-training discretisation, actual serialisation is solved for the measured artefact, and direct execution is feasible, while broader seeds, calibration distributions, and kernel optimisation remain open.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 11 of its 13 references that are in the map. First posted on arXiv 8 Sep 2026.

Builds on

11 of 13 references.

References · 11

  1. 2017Attention Is All You Needcited by 59,932
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  4. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  5. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 403
  6. 2024The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bitscited by 289
  7. 2024Extreme Compression of Large Language Models via Additive Quantizationcited by 223
  8. 2023BitNet: Scaling 1-bit Transformers for Large Language Modelscited by 175
  9. 2025PTQTP: Post-Training Quantization to Trit-Planes for Large Language Modelscited by 9
  10. 2026TWLA: Achieving Ternary Weights and Low-Bit Activations for LLMs via Post-Training Quantizationcited by 7
  11. 2026CAT-Q: Cost-efficient and Accurate Ternary Quantization for LLMscited by 5

Built on by

Not cited yet.