mlsanityData to 14 Oct 2026

DAST: Difficulty-Aware Self-Training on Large Language Models

arXiv 2503.09029 ↗First posted 12 Mar 2025cs.CLcited by 3

On the map: Synthetic data and self-improvement for LLMs → Self-rewarding alignment and self-training

Abstract (arXiv)

Present Large Language Models (LLM) self-training methods always under-sample on challenging queries, leading to inadequate learning on difficult problems which limits LLMs' ability. Therefore, this work proposes a difficulty-aware self-training (DAST) framework that focuses on improving both the quantity and quality of self-generated responses on challenging queries during self-training. DAST is specified in three components: 1) sampling-based difficulty level estimation, 2) difficulty-aware data augmentation, and 3) the self-training algorithm using SFT and DPO respectively. Experiments on mathematical tasks demonstrate the effectiveness and generalization of DAST, highlighting the critical role of difficulty-aware strategies in advancing LLM self-training.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Self-rewarding alignment and self-training, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 28 of its 41 references that are in the map. First posted on arXiv 12 Mar 2025.

Builds on

28 of 41 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Mitigating Tail Narrowing in LLM Self-Improvement via Socratic-Guided Sampling1 Nov
  2. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters6 Aug
  3. 2023Reinforced Self-Training (ReST) for Language Modeling17 Aug

Other references · 25

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2020Scaling Laws for Neural Language Modelscited by 6,586
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  7. 2024A Survey on LLM-as-a-Judgecited by 1,423
  8. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  9. 2016Training Deep Nets with Sublinear Memory Costcited by 848
  10. 2022Large Language Models Can Self-Improvecited by 664
  11. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 625
  12. 2025Demystifying Long Chain-of-Thought Reasoning in LLMscited by 326
13 more
  1. 2024V-STaR: Training Verifiers for Self-Taught Reasonerscited by 217
  2. 2024MathScale: Scaling Instruction Tuning for Mathematical Reasoningcited by 144
  3. 2020Training Question Answering Models From Synthetic Datacited by 95
  4. 2023Overthinking the Truth: Understanding how Language Models Process False Demonstrationscited by 81
  5. 2024Learning How Hard to Think: Input-Adaptive Allocation of LM Computationcited by 70
  6. 2020Zero-Shot Transfer Learning with Synthesized Data for Multi-Domain Dialogue State Trackingcited by 65
  7. 2024Key-Point-Driven Data Synthesis with its Enhancement on Mathematical Reasoningcited by 58
  8. 2023MuggleMath: Assessing the Impact of Query and Response Augmentation on Math Reasoningcited by 45
  9. 2023Joint Prompt Optimization of Stacked LLMs using Variational Inferencecited by 32
  10. 2024Self-DC: When to Reason and When to Act? Self Divide-and-Conquer for Compositional Unknown Questionscited by 30
  11. 2024Self-Training with Direct Preference Optimization Improves Chain-of-Thought Reasoningcited by 26
  12. 2023A Survey of the Evolution of Language Model-Based Dialogue Systems: Data, Task and Modelscited by 23
  13. 2024Extending Multilingual Speech Synthesis to 100+ Languages without Transcribed Datacited by 9

Built on by

Cited by year

Citing papers per year: 2025: 3.

3 papers.

  1. 2025HS-STaR: Hierarchical Sampling for Self-Taught Reasoners via Difficulty Estimation and Budget Reallocationcited by 18
  2. 2025ScaleDiff: Scaling Difficult Problems for Advanced Mathematical Reasoningcited by 6
  3. 2025Counteracting Matthew Effect in Self-Improvement of LVLMs through Head-Tail Re-balancingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Estimating problem difficulty without ground truth using Large Language Model comparisons
  2. 2025The LLM Already Knows: Estimating LLM-Perceived Question Difficulty via Hidden Representations
  3. 2025Theoretical Modeling of Large Language Model Self-Improvement Training Dynamics Through Solver-Verifier Gap
  4. 2025DeepDistill: Enhancing LLM Reasoning Capabilities via Large-Scale Difficulty-Graded Data Training
  5. 2025Continuous Self-Improvement of Large Language Models by Test-time Training with Verifier-Driven Sample Selection
  6. 2025The Path of Self-Evolving Large Language Models: Achieving Data-Efficient Learning via Intrinsic Feedback
  7. 2024AutoDetect: Towards a Unified Framework for Automated Weakness Detection in Large Language Models
  8. 2026Breaking the Pre-Sampling Barrier: Activation-Informed Difficulty-Aware Self-Consistency