mlsanityData to 7 Oct 2026

Random Recursive Models

arXiv 2610.00541 ↗First posted 30 Sep 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Recursive models create computational depth through parameter reuse, offering a parameter-efficient alternative to increasing model size. However, most recursive models repeatedly apply one learned transformation or a prescribed sequence of transformations, restricting computation to a fixed layer order. We introduce the Random Recursive Model (RRM), which maintains a pool of L learned layers and performs T recursive steps by sampling one layer independently with replacement for each example and step. This enables flexible layer reuse while retaining the parameter efficiency of recurrence. We evaluate RRM on challenging reasoning tasks, where it matches or exceeds the baselines, often with 50-75 % fewer parameters. RRM can vary its depth at inference, including beyond that seen during training, without retraining or adding parameters, improving tasks that benefit from deeper iterative computation. RRM also supports Monte Carlo inference and probabilistic test-time scaling, both of which improve performance without retraining. These insights may open new directions in neural network architecture design.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 41 of its 49 references that are in the map. First posted on arXiv 30 Sep 2026.

Builds on

41 of 49 references.

References · 41

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  3. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  4. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  5. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  6. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  7. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
  8. 2016Deep Networks with Stochastic Depthcited by 973
  9. 2018AutoAugment: Learning Augmentation Policies from Datacited by 728
  10. 2022Mixture-of-Experts with Expert Choice Routingcited by 611
  11. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 591
  12. 2016Adaptive Computation Time for Recurrent Neural Networkscited by 479
29 more
  1. 2019Reducing Transformer Depth on Demand with Structured Dropoutcited by 434
  2. 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approachcited by 372
  3. 2017PathNet: Evolution Channels Gradient Descent in Super Neural Networkscited by 362
  4. 2017SkipNet: Learning Dynamic Routing in Convolutional Networkscited by 328
  5. 2023From Sparse to Soft Mixtures of Expertscited by 245
  6. 2016FractalNet: Ultra-Deep Neural Networks without Residualscited by 216
  7. 2017BlockDrop: Dynamic Inference Paths in Residual Networkscited by 185
  8. 2025Hierarchical Reasoning Modelcited by 167
  9. 2015Conditional Computation in Neural Networks for faster modelscited by 166
  10. 2021PonderNet: Learning to Pondercited by 156
  11. 2025Scaling Latent Reasoning via Looped Language Modelscited by 145
  12. 2023Looped Transformers are Better at Learning Learning Algorithmscited by 135
  13. 2017Routing Networks: Adaptive Selection of Non-linear Functions for Multi-Task Learningcited by 125
  14. 2025Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computationcited by 120
  15. 2015Neural GPUs Learn Algorithmscited by 118
  16. 2021Can You Learn an Algorithm? Generalizing from Easy to Hard Problems with Recurrent Networkscited by 108
  17. 2019Exploring Randomly Wired Neural Networks for Image Recognitioncited by 103
  18. 2024Looped Transformers for Length Generalizationcited by 91
  19. 2024Relaxed Recursive Transformers: Effective Parameter Sharing with Layer-wise LoRAcited by 72
  20. 2017Recurrent Relational Networkscited by 71
  21. 2018Modular Networks: Learning to Decompose Neural Computationcited by 62
  22. 2025Inner Thinking Transformer: Leveraging Dynamic Depth Scaling to Foster Adaptive Internal Thinkingcited by 56
  23. 2026LoopFormer: Elastic-Depth Looped Transformers for Latent Reasoning via Shortcut Modulationcited by 48
  24. 2024Causal Language Modeling Can Elicit Search and Reasoning Capabilities on Logic Puzzlescited by 35
  25. 2022End-to-end Algorithm Synthesis with Recurrent Networks: Logical Extrapolation Without Overthinkingcited by 33
  26. 2016Swapout: Learning an ensemble of deep architecturescited by 19
  27. 2026Probabilistic Tiny Recursive Modelcited by 16
  28. 2025Skip a Layer or Loop it? Test-Time Depth Adaptation of Pretrained LLMscited by 14
  29. 2026Universal Transformers Need Memory: Depth-State Trade-offs in Adaptive Recursive Reasoningcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Form Follows Function: Recursive Stem Model
  2. 2026What Limits Recursive Reasoning Models: Optimization, Architecture and Test-Time Scaling
  3. 2026Generative Recursive Reasoning
  4. 2026Energy-guided Recursive Model
  5. 2025Less is More: Recursive Reasoning with Tiny Networks
  6. 2026Recursive Vision Language Models for General Symbolic Reasoning
  7. 2026Boosting Inference with Guided Reasoning: Stochastic Exploration for Recursive Models
  8. 2026Recirculation