mlsanityData to 7 Oct 2026

The Multiple Ticket Hypothesis: Random Sparse Subnetworks Suffice for RLVR

arXiv 2602.01599 ↗First posted 2 Feb 2026cs.LG cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

The Lottery Ticket Hypothesis demonstrated that sparse subnetworks can match full-model performance, suggesting parameter redundancy. Meanwhile, in Reinforcement Learning with Verifiable Rewards (RLVR), recent work has shown that updates concentrate on a sparse subset of parameters, which further lends evidence to this underlying redundancy. We study the simplest possible way to exploit this redundancy: training only a randomly selected subset of parameters at extreme sparsities. Empirically, we find that training just 1% of parameters matches or exceeds full-parameter RLVR finetuning across 3 models and 2 task domains. Moreover, different random masks show minimal overlap (≤ 0.005 Jaccard similarity) and yet all succeed, suggesting pretrained models contain many viable sparse subnetworks rather than one privileged set. We term this the Multiple Ticket Hypothesis. We explain this phenomenon through the implicit per-step KL constraint in RLVR, which restricts updates to a low-dimensional subspace, enabling arbitrary sparse masks to succeed.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 46 of its 62 references that are in the map. First posted on arXiv 2 Feb 2026.

Builds on

46 of 62 references.

References · 46

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2017Proximal Policy Optimization Algorithmscited by 14,441
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  7. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  8. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  9. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  10. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  11. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  12. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
34 more
  1. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  2. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  3. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  4. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  5. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  6. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  7. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,276
  8. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,234
  9. 2023PaLM 2 Technical Reportcited by 1,154
  10. 2025Group Sequence Policy Optimizationcited by 788
  11. 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 784
  12. 2020Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuningcited by 658
  13. 2018Gradient Descent Provably Optimizes Over-parameterized Neural Networkscited by 655
  14. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 622
  15. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wildcited by 505
  16. 2024Training Language Models to Self-Correct via Reinforcement Learningcited by 405
  17. 2020Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stoppingcited by 377
  18. 2023Fine-Tuning Language Models with Just Forward Passescited by 337
  19. 2023Multimodal Foundation Models: From Specialists to General-Purpose Assistantscited by 247
  20. 2020Proving the Lottery Ticket Hypothesis: Pruning is All You Needcited by 200
  21. 2024ReFT: Representation Finetuning for Language Modelscited by 186
  22. 2025RL's Razor: Why Online Reinforcement Learning Forgets Lesscited by 162
  23. 2023Task-Specific Skill Localization in Fine-tuned Language Modelscited by 104
  24. 2019Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLPcited by 73
  25. 2022The State of Sparse Training in Deep Reinforcement Learningcited by 60
  26. 2025The Path Not Taken: RLVR Provably Learns Off the Principalscited by 52
  27. 2025RandLoRA: Full-rank parameter-efficient fine-tuning of large modelscited by 34
  28. 2025RL-PLUS: Countering Capability Boundary Collapse of LLMs in Reinforcement Learning with Hybrid-policy Optimizationcited by 34
  29. 2024Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMscited by 31
  30. 2024Scaling Sparse Fine-Tuning to Large Language Modelscited by 25
  31. 2021The Elastic Lottery Ticket Hypothesiscited by 23
  32. 2021On Lottery Tickets and Minimal Task Representations in Deep Reinforcement Learningcited by 23
  33. 2024Random Masking Finds Winning Tickets for Parameter Efficient Fine-tuningcited by 16
  34. 2019Ctrl-Z: Recovering from Instability in Reinforcement Learningcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Preference Tuning as Spectral Update Reorganizationcited by 0
  2. 2026Understanding LLM Parameter Update Sparsity through the Lens of Fishercited by 0