mlsanityData to 14 Oct 2026

Low-Bit Recurrent States in Hybrid Language Models

arXiv 2609.30950 ↗First posted 25 Sep 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Hybrid language models maintain fixed-size recurrent states, but existing quantizers typically use eight bits or more. Quantization errors persist according to channel decay rates. We derive distortion weights from the observability Gramian and combine them with normalized state ranges for mixed-precision bit allocation, without calibration data, rotation, or training. We also quantize decay rates logarithmically. With per-token state quantization, a four-bit mean payload reduces excess negative log-likelihood by factors of 3.3--27.9 relative to the best of seven baselines across three hybrid models; metadata costs vary. At six bits, negative log-likelihood differs from the FP32-state baseline by less than 0.005 nats. Ablations separate gains from variable bit widths, decay weighting, and range normalization. With less frequent write-backs, gains diminish and depend on the model and budget.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 28 of its 34 references that are in the map. First posted on arXiv 25 Sep 2026.

Builds on

28 of 34 references.

References · 28

  1. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  2. 2016Pointer Sentinel Mixture Modelscited by 3,096
  3. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
  4. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,336
  5. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 637
  6. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
  7. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 591
  8. 2024Gated Delta Networks: Improving Mamba2 with Delta Rulecited by 532
  9. 2025Kimi Linear: An Expressive, Efficient Attention Architecturecited by 196
  10. 2023Microscaling Data Formats for Deep Learningcited by 152
  11. 2025TurboQuant: Online Vector Quantization with Near-optimal Distortion Ratecited by 125
  12. 2025REAP the Experts: Why Pruning Prevails for One-Shot MoE compressioncited by 44
16 more
  1. 2026Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoningcited by 42
  2. 2026Nemotron 3 Super: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoningcited by 38
  3. 2025Four Over Six: More Accurate NVFP4 Quantization with Adaptive Block Scalingcited by 35
  4. 2026Gated DeltaNet-2: Decoupling Erase and Write in Linear Attentioncited by 33
  5. 2025Quamba2: A Robust and Scalable Post-training Quantization Framework for Selective State Space Modelscited by 16
  6. 2025KV Cache Transform Coding for Compact Storage in LLM Inferencecited by 13
  7. 2025Nemotron Elastic: Towards Efficient Many-in-One Reasoning LLMscited by 6
  8. 2026KVarN: Variance-Normalized KV-Cache Quantization Mitigates Error Accumulation in Reasoning Taskscited by 6
  9. 2026DAMP: Decay-Aware Mixed-Precision Recurrent-State Quantizationcited by 6
  10. 2026OScaR: The Occam's Razor for Extreme KV Cache Quantization in LLMs and Beyondcited by 5
  11. 2026KVBuffer: IO-aware Serving for Linear Attentioncited by 3
  12. 2026SSDi8: Accurate and Efficient 8-bit Quantization for State Space Dualitycited by 2
  13. 2026DASC: Decay-Aware State Compression for Hybrid Linear-Attention Servingcited by 2
  14. 2026Why Gated DeltaNet Survives 4-Bit Quantization: NVFP4 W4A4 for the Recurrent Half of a Hybrid 27B LLMcited by 2
  15. 2026KV Cache Compression Through the Lens of Transform Codingcited by 1
  16. 2026DeltaLog: Deferred Materialization of Recurrent States for Linear Attention Decodingcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Cherry on Top: Parameter Heterogeneity and Quantization in Large Language Models
  2. 2026The Multilingual Quantization Tax: Structural Collapse and Typological Fragility in Edge SLMs
  3. 2026Reliability Scaling Laws for Quantized Large Language Models
  4. 2026Quantization Degradation in Large Language Models: A Signal-Noise Perspective
  5. 2025An Extra RMSNorm is All You Need for Fine Tuning to 1.58 Bits
  6. 2025Mixed-Precision Quantization for Language Models: Techniques and Prospects
  7. 2024What Makes Quantization for Large Language Models Hard? An Empirical Study from the Lens of Perturbation
  8. 2026ARCHead: Activation-Metric Residual Correction for Large Language Model Output Heads