mlsanityData to 7 Oct 2026

Speed is Confidence

arXiv 2601.19085 ↗First posted 27 Jan 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Biological neural systems must be fast but are energy-constrained. Evolution's solution: act on the first signal. Winner-take-all circuits and time-to-first-spike coding implicitly treat when a neuron fires as an expression of confidence. We apply this principle to ensembles of Tiny Recursive Models (TRM) [Jolicoeur-Martineau et al., 2025]. On Sudoku-Extreme, halt-first selection achieves 97% accuracy vs. 91% for probability averaging -- while requiring 10x fewer reasoning steps. A single baseline model achieves 85.5% +/- 1.3%. Can we internalize this as a training-only cost? Yes: by maintaining K=4 parallel latent states but backpropping only through the lowest-loss "winner," we achieve 96.9% +/- 0.6% accuracy -- matching ensemble performance at 1x inference cost, with less than half the variance of the baseline. A key diagnostic: 89% of baseline failures are selection problems, revealing a 99% accuracy ceiling. As in nature, this work was also resource constrained: all experiments used a single RTX 5090. A modified SwiGLU [Shazeer, 2020] made Muon [Jordan et al., 2024] and high LR viable, enabling baseline training in 48 minutes and full WTA (K=4) in 6 hours on consumer hardware.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 12 of its 28 references that are in the map. First posted on arXiv 27 Jan 2026.

Builds on

12 of 28 references.

References · 12

  1. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  2. 2020GLU Variants Improve Transformercited by 1,724
  3. 2019Generating Long Sequences with Sparse Transformerscited by 1,523
  4. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
  5. 2018Universal Transformerscited by 613
  6. 2016Adaptive Computation Time for Recurrent Neural Networkscited by 479
  7. 2021PonderNet: Learning to Pondercited by 156
  8. 2020T2FSNN: Deep Spiking Neural Networks with Time-to-first-spike Codingcited by 45
  9. 2024PolyNet: Learning Diverse Solution Strategies for Neural Combinatorial Optimizationcited by 35
  10. 2018DESlib: A Dynamic ensemble selection library in Pythoncited by 18
  11. 2023Window-Based Early-Exit Cascades for Uncertainty Estimation: When Deep Ensembles are More Efficient than Single Modelscited by 15
  12. 2025Latent Reasoning in TRMs is Secretly a Policy Improvement Operatorcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026C-voting: Confidence-Based Test-Time Voting without Explicit Energy Functions
  2. 2024Performance Control in Early Exiting to Deploy Large Models at the Same Cost of Smaller Ones
  3. 2026Affinity Is Not Enough: Recovering the Free Energy Principle in Mixture-of-Experts
  4. 2025Bridging Efficiency and Safety: Formal Verification of Neural Networks with Early Exits
  5. 2026Computation-Aware Kalman Filtering with Model Selection for Neural Dynamics
  6. 2025Mixture of Neuron Experts
  7. 2026Let It Go or Learn to Self-Correct: Continuous Diffusion for Constrained Discrete Tasks
  8. 2025Do LLMs Signal When They're Right? Evidence from Neuron Agreement