mlsanityData to 14 Oct 2026

Omni-Thinker: Scaling Multi-Task RL in LLMs with Hybrid Reward and Task Scheduling

arXiv 2507.14783 ↗First posted 20 Jul 2025cs.LG cs.AIcited by 10

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

The pursuit of general-purpose artificial intelligence depends on large language models (LLMs) that can handle both structured reasoning and open-ended generation. We present Omni-Thinker, a unified reinforcement learning (RL) framework that scales LLMs across diverse tasks by combining hybrid rewards with backward-transfer-guided scheduling. Hybrid rewards integrate rule-based verifiable signals with preference-based evaluations from an LLM-as-a-Judge, enabling learning in both deterministic and subjective domains. Our scheduler orders tasks according to accuracy backward transfer (BWT), reducing forgetting and improving multi-task performance. Experiments across four domains show gains of 6.2% over joint training and 12.4% over model merging. Moreover, we demonstrate that simple assumptions on accuracy transfer yield accurate predictions of curriculum outcomes, with entropy dynamics explaining deviations due to generative tasks. These findings underscore the importance of BWT-aware scheduling and hybrid supervision for scaling RL-based post-training toward general-purpose LLMs.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 22 of its 48 references that are in the map. First posted on arXiv 20 Jul 2025.

Builds on

22 of 48 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025Magistral12 Jun
  2. 2025Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning2 Jun
  3. 2025The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models28 May
  4. 2025The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoning21 May
  5. 2025General-Reasoner: Advancing LLM Reasoning Across All Domains20 May
  6. 2025X-Reasoner: Towards Generalizable Reasoning Across Modalities and Domains6 May
  7. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Training22 Nov
  8. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 14

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2024GPT-4o System Cardcited by 4,588
  5. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  6. 2024DeepSeek-V3 Technical Reportcited by 811
  7. 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 546
  8. 2023Scaling Relationship on Learning Mathematical Reasoning with Large Language Modelscited by 427
  9. 2023How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Compositioncited by 223
  10. 2024MARIO: MAth Reasoning with code Interpreter Output -- A Reproducible Pipelinecited by 81
  11. 2025SuperGPQA: Scaling LLM Evaluation across 285 Graduate Disciplinescited by 79
  12. 2025JudgeLRM: Large Reasoning Models as a Judgecited by 78
2 more
  1. 2025Command A: An Enterprise-Ready Large Language Modelcited by 59
  2. 2024Enhancing Logical Reasoning in Large Language Models through Graph-based Synthetic Datacited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 8.

10 papers.

  1. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 180
  2. 2025Reinforcement Learning with Rubric Anchorscited by 113
  3. 2026Multi-Task GRPO: Reliable LLM Reasoning Across Taskscited by 14
  4. 2026Advancing General-Purpose Reasoning Models with Modular Gradient Surgerycited by 9
  5. 2026Transferability for General Reasoning: An Automated Curriculum for Multi-Domain RLVRcited by 5
  6. 2026SPARD: Self-Paced Curriculum for RL Alignment via Integrating Reward Dynamics and Data Utilitycited by 4
  7. 2026Aligning Large Language Models with Searcher Preferencescited by 2
  8. 2026SFT Conflicts, RL Coexists: A Theoretical and Empirical Analysis of Multi-Task Learning for LLMscited by 2
  9. 2026Prompt-Level Reward Specifications for Open-Ended Post-Trainingcited by 1
  10. 2026LLM Agents for Automated Post-Training via Structured Recipe Optimizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Scaling Up RL: Unlocking Diverse Reasoning in LLMs via Prolonged Training
  2. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMs
  3. 2025Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecycle
  4. 2025Tool Zero: Training Tool-Augmented LLMs via Pure RL from Scratch
  5. 2026SortedRL: Accelerating RL Training for LLMs through Online Length-Aware Scheduling
  6. 2026Learning at the Right Pace: Adaptive Data Scheduling Improves LLM Reinforcement Learning
  7. 2026The Art of Efficient Reasoning: Data, Reward, and Optimization
  8. 2025Reshaping Reasoning in LLMs: A Theoretical Analysis of RL Training Dynamics through Pattern Selection