Depth Laws for the Precision Floor of Trained Neural Networks: Amplification, Residual Scaling, and a Quantization-Aware Training Paradox

How many bits does a network need before its accuracy collapses, and how does this grow with depth? We study the precision floor, the perturbation level or bit-width at which accuracy falls halfway to chance, in MLPs, CNNs, Vision Transformers and nine pretrained language models, under post-training quantization (PTQ) and quantization- or noise-aware training (QAT). (i) A first-order theory sets the floor through one full-precision quantity, the predictive amplification G: ηc=Lambda/G, and G2 grows linearly in depth at a rate proportional to the squared residual branch scale. (ii) The predicted equality αPTQ=ρ of depth exponents holds within 95% intervals in twelve of thirteen trained architectures and in GPT-2 from 12 to 48 layers, with Lambda=1.45±14% across trained architectures. (iii) Residual branches scaled by 1/√D and pre-normalisation remove the depth penalty, and each quantizer turns noise into bits at a rate fixed by its step rule, giving bc=(α/γ)log2 D+C. (iv) A QAT paradox: noise-aware training roughly doubles the tolerable noise of shallow networks, but the gain decays with depth, so the depth law steepens (αQAT/αPTQ=1.45-1.47 on two datasets, ten seeds each). Decision margins, cross-layer error cancellation and heavy tails do not set the floor.