What Survives When You Compress a Recursive Reasoner for the Edge?
On the map: LLM reasoning → Test-time compute and efficient thinking
Abstract (arXiv)
Recursive reasoning models solve hard structured tasks with a few million parameters by iterating a latent state, but deploying them on edge hardware means compressing them -- and quantization noise compounds across recursive cycles rather than accumulating over output tokens, so single-pass intuitions fail. Here, we ask what survives. Across a full precision sweep and three tasks, aggressive compression preserves local prediction but destroys global reasoning: pruning, distillation, and linear attention drive puzzle-exact accuracy to zero while cell accuracy holds. Naive INT4 is different: its damage is architectural, striking MLP-mixing recursion but not attention on the same task, and per-channel calibration reverses it without retraining, where quantization-aware training does not. We also introduce carry-trajectory fidelity, the cosine similarity to the full-precision reasoning path, as a label-free signal that predicts both the damage and the recovery before any task evaluation, with a threshold that transfers across tasks. The result is a deployment recipe: flash-streamed embeddings remove a 99.4 MB bottleneck, one recursive cycle matches full-depth accuracy at 6x fewer FLOPs, and calibrated INT4 brings the backbone to 3.3 MB, inside a 4 MB budget.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 20 of its 28 references that are in the map. First posted on arXiv 25 Jun 2026.
Builds on
20 of 28 references.
Closest ancestors · 5
Same direction, newest first
- 2026What Makes Low-Bit Quantization-Aware Training Work for Reasoning LLMs? A Systematic Study21 Jan
- 2025Latent Thinking Optimization: Your Latent Reasoning Language Model Secretly Encodes Reward Signals in Its Latent Thoughts30 Sep
- 2025Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Models7 Apr
- 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach7 Feb
- 2022End-to-end Algorithm Synthesis with Recurrent Networks: Logical Extrapolation Without Overthinking11 Feb
Other references · 15
- 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
- 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
- 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 763
- 2018Universal Transformerscited by 613
- 2021A White Paper on Neural Network Quantizationcited by 453
- 2020MCUNet: Tiny Deep Learning on IoT Devicescited by 269
- 2021I-BERT: Integer-only BERT Quantizationcited by 250
- 2023Do Emergent Abilities Exist in Quantized Large Language Models: An Empirical Studycited by 43
- 2024Tracing Representation Progression: Analyzing and Enhancing Layer-Wise Similaritycited by 36
- 2026Loop, Think, & Generalize: Implicit Reasoning in Recurrent-Depth Transformerscited by 34
- 2024QT-DoG: Quantization-aware Training for Domain Generalizationcited by 13
- 2022QReg: On Regularization Effects of Quantizationcited by 9
3 more
- 2024QGen: On the Ability to Generalize in Quantization Aware Trainingcited by 7
- 2025Neural Network Quantization for Microcontrollers: A Comprehensive Survey of Methods, Platforms, and Applicationscited by 6
- 2023Fighting over-fitting with quantization for learning deep neural networks on noisy labelscited by 1
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026Q-DEQ: Discrete Solving and Quantization for Deep Equilibrium Models in Time Series Forecasting under Edge Deployment Coding Constraintscited by 0
- 2026Quantizing Looped Transformers: Feedback Exposure and Calibration Blindnesscited by 0
- 2026A Tilted Bowl Is Not a Slippery Slope: Compressing Looped Modelscited by 0
- 2026A Tilted Bowl Is Not a Slippery Slope: Compressing Looped Models30 Sep 2026
- 2026Quantizing Looped Transformers: Feedback Exposure and Calibration Blindness25 Sep 2026
- 2026Q-DEQ: Discrete Solving and Quantization for Deep Equilibrium Models in Time Series Forecasting under Edge Deployment Coding Constraints21 Sep 2026
Similar, not cited
Close in wording, no citation either way
- 2026What to Preserve in Recursive Computation: A Local Predictive Sufficiency Principle
- 2026Memory Augmentation Unlocks Efficient Chain-of-Thought Reasoning
- 2025Reasoning Path Compression: Compressing Generation Trajectories for Efficient LLM Reasoning
- 2026Fewer Tokens, Smaller Cache: Reward-Coordinated Efficient Reasoning
- 2025ConCISE: Confidence-guided Compression in Step-by-step Efficient Reasoning
- 2026Quantizing Recursive Reasoning Models
- 2025Breadcrumbs Reasoning: Memory-Efficient Reasoning with Compression Beacons
- 2025When Reasoning Meets Compression: Understanding the Effects of LLMs Compression on Large Reasoning Models