# When Can Depth Replace Precision? A Resource Theory of Quantized Neural Computation

arXiv: https://arxiv.org/abs/2607.23390 · HTML: https://mlsanity.com/paper/2607.23390/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 107 references that are in the map.
- First posted on arXiv 25 Jul 2026.

## Builds on (most cited first, 30 of 31)

- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits](https://mlsanity.com/paper/2402.17764/) (2024)
- [BitNet b1.58 2B4T Technical Report](https://mlsanity.com/paper/2504.12285/) (2025)
- [Qronos: Correcting the Past by Shaping the Future... in Post-Training Quantization](https://mlsanity.com/paper/2505.11695/) (2025)
- [I-LLM: Efficient Integer-Only Inference for Fully-Quantized Low-Bit Large Language Models](https://mlsanity.com/paper/2405.17849/) (2024)
- [Provable Post-Training Quantization: Theoretical Analysis of OPTQ and Qronos](https://mlsanity.com/paper/2508.04853/) (2025)
- [GEMQ: Global Expert-Level Mixed-Precision Quantization for MoE LLMs](https://mlsanity.com/paper/2605.23078/) (2026)
- [AlphaQ: Calibration-Free Bit Allocation for Mixture-of-Experts Quantization](https://mlsanity.com/paper/2606.04980/) (2026)
- [Accumulator-Aware Post-Training Quantization for Large Language Models](https://mlsanity.com/paper/2409.17092/) (2024)
- [LoopQ: Quantization for Looped Language Models](https://mlsanity.com/paper/2605.16343/) (2026)
- [Value-and-Structure Alignment for Routing-Consistent Quantization of Mixture-of-Experts Models](https://mlsanity.com/paper/2606.05688/) (2026)
- [CAT-Q: Cost-efficient and Accurate Ternary Quantization for LLMs](https://mlsanity.com/paper/2606.26650/) (2026)
- [Three Quantization Regimes for ReLU Networks](https://mlsanity.com/paper/2405.01952/) (2024)
- [Improving Quantization with Post-Training Model Expansion](https://mlsanity.com/paper/2503.17513/) (2025)
- [ECO: Quantized Training without Full-Precision Master Weights](https://mlsanity.com/paper/2601.22101/) (2026)
- [On Expressive Power of Quantized Neural Networks under Fixed-Point Arithmetic](https://mlsanity.com/paper/2409.00297/) (2024)
- [The Quantization Benefits of Residual-Free Transformers](https://mlsanity.com/paper/2605.25880/) (2026)
- [The Expressive Power of Low Precision Softmax Transformers with (Summarized) Chain-of-Thought](https://mlsanity.com/paper/2605.18079/) (2026)
- [Sensitivity analysis of hybrid systems with state jumps with application to trajectory tracking](https://mlsanity.com/paper/1406.2811/) (2014)
- [Efficient Quantization of Mixture-of-Experts with Theoretical Generalization Guarantees](https://mlsanity.com/paper/2604.06515/) (2026)
- [RQ-MoE: Residual Quantization via Mixture of Experts for Efficient Input-Dependent Vector Compression](https://mlsanity.com/paper/2605.14359/) (2026)
- [Physics-Inspired Binary Neural Networks: Interpretable Compression with Theoretical Guarantees](https://mlsanity.com/paper/2502.01908/) (2025)
- [MGS: Markov Greedy Sums for Accurate Low-Bitwidth Floating-Point Accumulation](https://mlsanity.com/paper/2504.09072/) (2025)
- [SDQ-LLM: Sigma-Delta Quantization for 1-bit LLMs of any size](https://mlsanity.com/paper/2510.03275/) (2025)
- [Every Bit Counts: A Theoretical Study of Precision-Expressivity Tradeoffs in Quantized Transformers](https://mlsanity.com/paper/2602.02707/) (2026)
- [SimCert: Probabilistic Certification for Behavioral Similarity in Deep Neural Network Compression](https://mlsanity.com/paper/2603.14818/) (2026)
- [Featurized Occupation Measures for Structured Global Search in Numerical Optimal Control](https://mlsanity.com/paper/2603.16231/) (2026)
- [Expressive Power of Floating-Point Neural Networks with Arbitrary Reduction Orders and Inexact Activation Implementations](https://mlsanity.com/paper/2605.28704/) (2026)
- [DynamicPTQ: Mitigating Activation Quantization Collapse via Residual-Stream Dynamics](https://mlsanity.com/paper/2606.12487/) (2026)

## Built on by (most cited first, 1 of 1)

- [Depth Laws for the Precision Floor of Trained Neural Networks: Amplification, Residual Scaling, and a Quantization-Aware Training Paradox](https://mlsanity.com/paper/2609.32060/) (2026)

## Abstract (arXiv)

When can additional low-bit residual computation replace missing numerical precision for a fixed input-output map? We model a quantized residual system over a fixed horizon as a pure schedule selecting fields from a declared low-bit operation library, and use relaxed controls to characterize its infinite-depth limit. The distance from the target to the closed relaxed reachable set is the exact structural floor: no increase in depth can remove it for that library. Pure schedules approach the relaxed class at rate O(D-1) under bounded-variation time dependence and O(D-vartheta+D-1) under Holder dependence of exponent vartheta. Execution arithmetic can reverse this conclusion: full-state write-back introduces a Dρz penalty and can freeze residual updates, whereas increment error feedback replaces this growth by a bounded carry term and obeys an exact common-lattice conservation law. A fixed-teacher converse makes this rate sharp: for coherent depth-L first-order high-precision comparators, accuracy matching requires D=Theta(L). Learned codebooks add a metadata resource, while state-dependent routing introduces hybrid event conditions. Verified primal and dual bounds yield feasible, impossible, or unresolved decisions before training. Companion software implements the workflow, and Lean 4 machine-checks the exact discrete core. Depth replaces precision only relative to a declared library, horizon, execution semantics, and routing model.