mlsanityData to 14 Oct 2026

AdaSTaR: Adaptive Data Sampling for Training Self-Taught Reasoners

arXiv 2505.16322 ↗First posted 22 May 2025cs.LG cs.AI cs.CLcited by 6

On the map: LLM reasoning → Test-time compute and efficient thinking

also in 1 more
Abstract (arXiv)

Self-Taught Reasoners (STaR), synonymously known as Rejection sampling Fine-Tuning (RFT), is an integral part of the training pipeline of self-improving reasoning Language Models (LMs). The self-improving mechanism often employs random observation (data) sampling. However, this results in trained observation imbalance; inefficiently over-training on solved examples while under-training on challenging ones. In response, we introduce Adaptive STaR (AdaSTaR), a novel algorithm that rectifies this by integrating two adaptive sampling principles: (1) Adaptive Sampling for Diversity: promoting balanced training across observations, and (2) Adaptive Sampling for Curriculum: dynamically adjusting data difficulty to match the model's evolving strength. Across six benchmarks, AdaSTaR achieves best test accuracy in all instances (6/6) and reduces training FLOPs by an average of 58.6% against an extensive list of baselines. These improvements in performance and efficiency generalize to different pre-trained LMs and larger models, paving the way for more efficient and effective self-improving LMs.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 59 of its 75 references that are in the map. First posted on arXiv 22 May 2025.

Builds on

59 of 75 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025Nemotron-H: A Family of Accurate and Efficient Hybrid Mamba-Transformer Models4 Apr
  2. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models20 Mar
  3. 2025EXAONE Deep: Reasoning Enhanced Language Models16 Mar
  4. 2024B-STaR: Monitoring and Balancing Exploration and Exploitation in Self-Taught Reasoners23 Dec
  5. 2024Self-Training Meets Consistency: Improving LLMs' Reasoning with Consistency-Driven Rationale Evaluation10 Nov
  6. 2024ReGenesis: LLMs can Grow into Reasoning Generalists via Self-Improvement3 Oct
  7. 2024V-STaR: Training Verifiers for Self-Taught Reasoners9 Feb
  8. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Models17 May
  9. 2022STaR: Bootstrapping Reasoning With Reasoning28 Mar

Other references · 50

  1. 2017Attention Is All You Needcited by 60,361
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  5. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  6. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  7. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  8. 2020Scaling Laws for Neural Language Modelscited by 6,586
  9. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  10. 2024Qwen2.5 Technical Reportcited by 4,942
  11. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  12. 2023Let's Verify Step by Stepcited by 3,789
38 more
  1. 2023Qwen Technical Reportcited by 3,344
  2. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  3. 2022Emergent Abilities of Large Language Modelscited by 2,512
  4. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
  5. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,345
  6. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
  7. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 991
  8. 2022Solving math word problems with process- and outcome-based feedbackcited by 901
  9. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 878
  10. 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 807
  11. 2020Contrastive Learning with Hard Negative Samplescited by 628
  12. 2023Scaling Relationship on Learning Mathematical Reasoning with Large Language Modelscited by 427
  13. 2023Larger language models do in-context learning differentlycited by 394
  14. 2023Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Modelscited by 349
  15. 2022Large Language Models Are Reasoning Teacherscited by 329
  16. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 301
  17. 2019On The Power of Curriculum Learning in Training Deep Networkscited by 286
  18. 2024Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoningcited by 279
  19. 2021Scaling Laws for Transfercited by 272
  20. 2024Iterative Reasoning Preference Optimizationcited by 246
  21. 2025Inference-Time Scaling for Generalist Reward Modelingcited by 239
  22. 2022Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matterscited by 222
  23. 2023Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Lawscited by 159
  24. 2024QuRating: Selecting High-Quality Data for Training Language Modelscited by 157
  25. 2022Coverage-centric Coreset Selection for High Pruning Ratescited by 117
  26. 2023CLadder: Assessing Causal Reasoning in Language Modelscited by 107
  27. 2024GLoRe: When, Where, and How to Improve LLM Reasoning via Global and Local Refinementscited by 103
  28. 2024Systematic Evaluation of LLM-as-a-Judge in LLM Alignment Tasks: Explainable Metrics and Diverse Prompt Templatescited by 72
  29. 2024Lean-STaR: Learning to Interleave Thinking and Provingcited by 68
  30. 2025Z1: Efficient Test-time Scaling with Codecited by 35
  31. 2020DEAL: Difficulty-aware Active Learning for Semantic Segmentationcited by 32
  32. 2024AutoPSV: Automated Process-Supervised Verifiercited by 30
  33. 2024Dataset Decomposition: Faster LLM Training with Variable Sequence Length Curriculumcited by 26
  34. 2025Lightweight Dataset Pruning without Full Training via Example Difficulty and Prediction Uncertaintycited by 22
  35. 2024Enhancing Code Generation Performance of Smaller Models by Distilling the Reasoning Ability of LLMscited by 17
  36. 2024Automatic Curriculum Expert Iteration for Reliable LLM Reasoningcited by 16
  37. 2024Toward Adaptive Reasoning in Large Language Models with Thought Rollbackcited by 11
  38. 2024Rationale-Aware Answer Verification by Pairwise Self-Evaluationcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 3.

6 papers.

  1. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligencecited by 170
  2. 2026FailForge: Distilling Procedural Competence from Persistent Failures into Code Agentscited by 2
  3. 2025Aligning Reasoning LLMs for Materials Discovery with Physics-aware Rejection Samplingcited by 1
  4. 2026AnE: Pushing the Reasoning Frontier of Multimodal LLMs via Anchor Evolutioncited by 1
  5. 2026MMVistaReason: Toward Open-Data and Post-Training Recipes for Multimodal Reasoningcited by 1
  6. 2025Learning to Reason in LLMs by Expectation Maximizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025HS-STaR: Hierarchical Sampling for Self-Taught Reasoners via Difficulty Estimation and Budget Reallocation
  2. 2024RL-STaR: Theoretical Analysis of Reinforcement Learning Frameworks for Self-Taught Reasoner
  3. 2025STaR: Towards Effective and Stable Table Reasoning via Slow-Thinking Large Language Models
  4. 2024Star-Agents: Automatic Data Optimization with LLM Agents for Instruction Tuning
  5. 2025Learning to Reason Across Parallel Samples for LLM Reasoning
  6. 2025SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvement
  7. 2025Continuous Self-Improvement of Large Language Models by Test-time Training with Verifier-Driven Sample Selection
  8. 2026Ada-RS: Adaptive Rejection Sampling for Selective Thinking