mlsanityData to 7 Oct 2026

Plasticity vs. Rigidity: The Impact of Low-Rank Adapters on Reasoning on a Micro-Budget

arXiv 2601.06677 ↗First posted 10 Jan 2026cs.LG cs.AIcited by 1

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Recent advances in mathematical reasoning typically rely on massive scale, yet the question remains: can strong reasoning capabilities be induced in small language models (≤1.5B) under extreme constraints? We investigate this by training models on a single A40 GPU (48GB) for under 24 hours using Reinforcement Learning with Verifiable Rewards (RLVR) and Low-Rank Adaptation (LoRA). We find that the success of this ``micro-budget" regime depends critically on the interplay between adapter capacity and model initialization. While low-rank adapters (r=8) consistently fail to capture the complex optimization dynamics of reasoning, high-rank adapters (r=256) unlock significant plasticity in standard instruction-tuned models. Our best result achieved an impressive 40.0% Pass@1 on AIME 24 (an 11.1% absolute improvement over baseline) and pushed Pass@16 to 70.0%, demonstrating robust exploration capabilities. However, this plasticity is not universal: while instruction-tuned models utilized the budget to elongate their chain-of-thought and maximize reward, heavily math-aligned models suffered performance collapse, suggesting that noisy, low-budget RL updates can act as destructive interference for models already residing near a task-specific optimum.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 12 of its 15 references that are in the map. First posted on arXiv 10 Jan 2026.

Builds on

12 of 15 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Tina: Tiny Reasoning Models via LoRA22 Apr
  2. 2025s1: Simple test-time scaling31 Jan

Other references · 10

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  5. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  6. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,276
  7. 2023LIMA: Less Is More for Alignmentcited by 1,080
  8. 2025LIMO: Less is More for Reasoningcited by 488
  9. 2025The Entropy Mechanism of Reinforcement Learning for Reasoning Language Modelscited by 463
  10. 2025Reinforcement Learning for Reasoning in Small LLMs: What Works and What Doesn'tcited by 72

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Mask What Matters: Mitigating Object Hallucinations in Multimodal Large Language Models with Object-Aligned Visual Contrastive Decodingcited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2025From Efficiency to Adaptivity: A Deeper Look at Adaptive Reasoning in Large Language Models
  2. 2026Are More Tokens Rational? Inference-Time Scaling in Language Models as Adaptive Resource Rationality
  3. 2026Proof-Carrying Cognition: Closing the Verification Gap with Reality-Settled Reward
  4. 2026Revealing Behavioral Plasticity in Large Language Models: A Token-Conditional Perspective
  5. 2026ASTER: Agentic Scaling with Tool-integrated Extended Reasoning
  6. 2025Steering LLM Reasoning Through Bias-Only Adaptation
  7. 2026Knowing When Thinking Is Not Enough: Teaching Small Reasoning Models to Reason Beyond Their Parametric Knowledge
  8. 2026The Art of Efficient Reasoning: Data, Reward, and Optimization