mlsanityData to 7 Oct 2026

WIST: Web-Grounded Iterative Self-Play Tree for Domain-Targeted Reasoning Improvement

arXiv 2603.22352 ↗First posted 22 Mar 2026cs.LG cs.AIcited by 0

On the map: LLM reasoning → Reinforcement learning for reasoning

also in 1 more
Abstract (arXiv)

Recent progress in reinforcement learning with verifiable rewards (RLVR) offers a practical path to self-improvement of language models, but existing methods face a key trade-off: endogenous self-play can drift over iterations, while corpus-grounded approaches rely on curated data environments. We present WIST, a Web-grounded Iterative Self-play Tree framework for domain-targeted reasoning improvement that learns directly from the open web without requiring any pre-arranged domain corpus. WIST incrementally expands a domain tree for exploration, and retrieves and cleans path-consistent web corpus to construct a controllable training environment. It then performs Challenger--Solver self-play with verifiable rewards, and feeds learnability signals back to update node posteriors and guide subsequent exploration through an adaptive curriculum. Across four backbones, WIST consistently improves over the base models and typically outperforms both purely endogenous self-evolution and corpus-grounded self-play baselines, with the Overall gains reaching +9.8 (textitQwen3-4B-Base) and +9.7 (textitOctoThinker-8B). WIST is also domain-steerable, improving textitQwen3-8B-Base by +14.79 in medicine and textitQwen3-4B-Base by +5.28 on PhyBench. Ablations further confirm the importance of WIST's key components for stable open-web learning. Our Code is available at https://github.com/lfy-123/WIST.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 33 of its 51 references that are in the map. First posted on arXiv 22 Mar 2026.

Builds on

33 of 51 references.

Closest ancestors · 11

Same direction, newest first

  1. 2025OctoThinker: Mid-training Incentivizes Reinforcement Learning Scaling25 Jun
  2. 2025Consistent Paths Lead to Truth: Self-Rewarding Reinforcement Learning for LLM Reasoning10 Jun
  3. 2025Can Large Reasoning Models Self-Train?27 May
  4. 2025Learning to Reason without External Rewards26 May
  5. 2025General-Reasoner: Advancing LLM Reasoning Across All Domains20 May
  6. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Data6 May
  7. 2025Understanding R1-Zero-Like Training: A Critical Perspective26 Mar
  8. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild24 Mar
  9. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan
  10. 2024OpenRLHF: An Easy-to-use, Scalable and High-performance RLHF Framework20 May
  11. 2022Solving math word problems with process- and outcome-based feedback25 Nov

Other references · 22

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2025Qwen3 Technical Reportcited by 8,195
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  5. 2024GPT-4o System Cardcited by 4,467
  6. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  7. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 724
  8. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 552
  9. 2019Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialogcited by 283
  10. 2025R-Zero: Self-Evolving Reasoning LLM from Zero Datacited by 210
  11. 2025Llama-Nemotron: Efficient Reasoning Modelscited by 151
  12. 2019AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligencecited by 84
10 more
  1. 2025NaturalReasoning: Reasoning in the Wild with 2.8M Challenging Questionscited by 81
  2. 2025SPICE: Self-Play In Corpus Environments Improves Reasoningcited by 78
  3. 2025SuperGPQA: Scaling LLM Evaluation across 285 Graduate Disciplinescited by 67
  4. 2025PHYBench: Holistic Evaluation of Physical Perception and Reasoning in Large Language Modelscited by 57
  5. 2025Self-Questioning Language Modelscited by 47
  6. 2025Self-Improving Language Models for Evolutionary Program Synthesis: A Case Study on ARC-AGIcited by 34
  7. 2025Nemotron-CC-Math: A 133 Billion-Token-Scale High Quality Math Pretraining Datasetcited by 34
  8. 2025The Common Pile v0.1: An 8TB Dataset of Public Domain and Openly Licensed Textcited by 25
  9. 2025Condor: Enhance LLM Alignment with Knowledge-Driven Data Synthesis and Refinementcited by 17
  10. 2025Bohdi: Heterogeneous LLM Fusion with Automatic Data Explorationcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025SeRL: Self-Play Reinforcement Learning for Large Language Models with Limited Data
  2. 2025Towards Understanding Self-play for LLM Reasoning
  3. 2025Language Self-Play For Data-Free Training
  4. 2025OpenSIR: Open-Ended Self-Improving Reasoner
  5. 2025SERL: Self-Examining Reinforcement Learning on Open-Domain
  6. 2026Test-time Recursive Thinking: Self-Improvement without External Feedback
  7. 2026SAGE: Multi-Agent Self-Evolution for LLM Reasoning
  8. 2026SPADE: Self-Play in Adaptive Synthetic Executable Environments