# SPARD: Self-Paced Curriculum for RL Alignment via Integrating Reward Dynamics and Data Utility

arXiv: https://arxiv.org/abs/2604.07837 · HTML: https://mlsanity.com/paper/2604.07837/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 50 references that are in the map.
- First posted on arXiv 9 Apr 2026.

## Builds on (most cited first, 30 of 38)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [WildChat: 1M ChatGPT Interaction Logs in the Wild](https://mlsanity.com/paper/2405.01470/) (2024)
- [SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild](https://mlsanity.com/paper/2503.18892/) (2025)
- [From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipeline](https://mlsanity.com/paper/2406.11939/) (2024)
- [Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domains](https://mlsanity.com/paper/2507.17746/) (2025)
- [SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuning](https://mlsanity.com/paper/2408.05517/) (2024)
- [Skywork-Reward: Bag of Tricks for Reward Modeling in LLMs](https://mlsanity.com/paper/2410.18451/) (2024)
- [Inference-Time Scaling for Generalist Reward Modeling](https://mlsanity.com/paper/2504.02495/) (2025)
- [WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wild](https://mlsanity.com/paper/2406.04770/) (2024)
- [Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and Beyond](https://mlsanity.com/paper/2503.10460/) (2025)
- [RM-R1: Reward Modeling as Reasoning](https://mlsanity.com/paper/2505.02387/) (2025)
- [OpenRubrics: Towards Scalable Synthetic Rubric Generation for Reward Modeling and LLM Alignment](https://mlsanity.com/paper/2510.07743/) (2025)
- [Checklists Are Better Than Reward Models For Aligning Language Models](https://mlsanity.com/paper/2507.18624/) (2025)
- [DR Tulu: Reinforcement Learning with Evolving Rubrics for Deep Research](https://mlsanity.com/paper/2511.19399/) (2025)
- [Self-Evolving Curriculum for LLM Reasoning](https://mlsanity.com/paper/2505.14970/) (2025)
- [Writing-Zero: Bridge the Gap Between Non-verifiable Tasks and Verifiable Rewards](https://mlsanity.com/paper/2506.00103/) (2025)
- [DUMP: Automated Distribution-Level Curriculum Learning for RL-based LLM Post-training](https://mlsanity.com/paper/2504.09710/) (2025)
- [Entropy Law: The Story Behind Data Compression and LLM Performance](https://mlsanity.com/paper/2407.06645/) (2024)
- [QA-LIGN: Aligning LLMs through Constitutionally Decomposed QA](https://mlsanity.com/paper/2506.08123/) (2025)
- [Language Models that Think, Chat Better](https://mlsanity.com/paper/2509.20357/) (2025)
- [RAPID: Efficient Retrieval-Augmented Long Text Generation with Writing Planning and Information Discovery](https://mlsanity.com/paper/2503.00751/) (2025)
- [Thought-Augmented Planning for LLM-Powered Interactive Recommender Agent](https://mlsanity.com/paper/2506.23485/) (2025)
- [TD3: Tucker Decomposition Based Dataset Distillation Method for Sequential Recommendation](https://mlsanity.com/paper/2502.02854/) (2025)
- [Generative Large Recommendation Models: Emerging Trends in LLMs for Recommendation](https://mlsanity.com/paper/2502.13783/) (2025)
- [Killing Two Birds with One Stone: Unifying Retrieval and Ranking with a Single Generative Recommendation Model](https://mlsanity.com/paper/2504.16454/) (2025)
- [CoSteer: Collaborative Decoding-Time Personalization via Local Delta Steering](https://mlsanity.com/paper/2507.04756/) (2025)
- [RLMR: Reinforcement Learning with Mixed Rewards for Creative Writing](https://mlsanity.com/paper/2508.18642/) (2025)

## Built on by (most cited first, 4 of 4)

- [Learning from Emptiness: De-biasing Listwise Rerankers with Content-Agnostic Probability Calibration](https://mlsanity.com/paper/2604.10150/) (2026)
- [IE as Cache: Information Extraction Enhanced Agentic Reasoning](https://mlsanity.com/paper/2604.14930/) (2026)
- [AsymSpec: Context-Asymmetric Speculative Decoding for Agentic LLMs](https://mlsanity.com/paper/2608.26004/) (2026)
- [Rethinking the Necessity of Adaptive Retrieval-Augmented Generation through the Lens of Adaptive Listwise Ranking](https://mlsanity.com/paper/2604.15621/) (2026)

## Abstract (arXiv)

The evolution of Large Language Models (LLMs) is shifting the focus from single, verifiable tasks toward complex, open-ended real-world scenarios, imposing significant challenges on the post-training phase. In these settings, the scale and complexity of reward systems have grown significantly, transitioning toward multi-objective formulations that encompass a comprehensive spectrum of model capabilities and application contexts. However, traditional methods typically rely on fixed reward weights, ignoring non-stationary learning dynamics and struggling with data heterogeneity across dimensions. To address these issues, we propose SPARD, a framework that establishes an automated, self-paced curriculum by perceiving learning progress to dynamically adjust multi-objective reward weights and data importance, thereby synchronizing learning intent with data utility for optimal performance. Extensive experiments across multiple benchmarks demonstrate that SPARD significantly enhances model capabilities across all domains.