mlsanityData to 14 Oct 2026

Nemotron-CrossThink: Scaling Self-Learning beyond Math Reasoning

arXiv 2504.13941 ↗First posted 15 Apr 2025cs.LG cs.AIcited by 31

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Large Language Models (LLMs) have shown strong reasoning capabilities, particularly when enhanced through Reinforcement Learning (RL). While prior work has successfully applied RL to mathematical reasoning -- where rules and correctness are well-defined -- generalizing these methods to broader reasoning domains remains challenging due to limited data, the lack of verifiable reward structures, and diverse task requirements. In this work, we propose NEMOTRON-CROSSTHINK, a framework that systematically incorporates multi-domain corpora, including both synthetic and real-world question-answer pairs, into RL training to improve generalization across diverse reasoning tasks. NEMOTRON-CROSSTHINK addresses key challenges by (1) incorporating data from varied sources spanning STEM, humanities, social sciences, etc.; (2) applying structured templates (e.g., multiple-choice and open-ended) to control answer-space complexity; (3) filtering for verifiable answers; and (4) optimizing data blending strategies that utilizes data from multiple sources effectively. Our approach enables scalable and verifiable reward modeling beyond mathematics and demonstrates improved accuracies on both math (MATH-500: +30.1%, AMC23:+27.5%) and non-math reasoning benchmarks (MMLU-PRO: +12.8%, GPQA-DIAMOND: +11.3%, AGIEVAL: +15.1%, SUPERGPQA: +3.8%). Moreover, NEMOTRON-CROSSTHINK exhibits significantly improved response efficiency -- using 28% fewer tokens for correct answers -- highlighting more focused and effective reasoning. Through NEMOTRON-CROSSTHINK, we demonstrate that integrating multi-domain, multi-format data in RL leads to more accurate, efficient, and generalizable LLMs.

Key facts. Cited by 31 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 33 of its 57 references that are in the map. First posted on arXiv 15 Apr 2025.

Builds on

33 of 57 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model31 Mar
  2. 2025Crossing the Reward Bridge: Expanding RL with Verifiable Rewards Across Diverse Domains31 Mar
  3. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild24 Mar
  4. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  5. 2025Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning20 Feb
  6. 2025Process Reinforcement through Implicit Rewards3 Feb
  7. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan
  8. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMs22 Jan
  9. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 24

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  3. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  4. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  5. 2024Mixtral of Expertscited by 1,823
  6. 2025s1: Simple test-time scalingcited by 1,341
  7. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  8. 2025LIMO: Less is More for Reasoningcited by 498
  9. 2024Scaling Synthetic Data Creation with 1,000,000,000 Personascited by 417
  10. 2024InternLM2 Technical Reportcited by 397
  11. 2025L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learningcited by 384
  12. 2025Demystifying Long Chain-of-Thought Reasoning in LLMscited by 326
12 more
  1. 2025SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Modelscited by 212
  2. 2023Rethinking Benchmark and Contamination for Language Models with Rephrased Samplescited by 195
  3. 2025Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and Beyondcited by 175
  4. 2024Nemotron-4 340B Technical Reportcited by 144
  5. 2024OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Datasetcited by 143
  6. 2024O1 Replication Journey: A Strategic Progress Report -- Part 1cited by 143
  7. 2024Language Models Learn to Mislead Humans via RLHFcited by 124
  8. 2025Nemotron-H: A Family of Accurate and Efficient Hybrid Mamba-Transformer Modelscited by 100
  9. 2025NaturalReasoning: Reasoning in the Wild with 2.8M Challenging Questionscited by 84
  10. 2025SuperGPQA: Scaling LLM Evaluation across 285 Graduate Disciplinescited by 79
  11. 2025Concise Reasoning via Reinforcement Learningcited by 79
  12. 2025O1 Replication Journey -- Part 3: Inference-time Scaling for Medical Reasoningcited by 35

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 19; 2026: 12.

31 papers.

Sort:

By area:

  1. 2025General-Reasoner: Advancing LLM Reasoning Across All Domainscited by 135
  2. 2025Reinforcement Learning Optimization for Large-Scale Learning: An Efficient and User-Friendly Scaling Librarycited by 101
  3. 2025Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspectivecited by 76
  4. 2025Training a Scientific Reasoning Model for Chemistrycited by 56
  5. 2025Front-Loading Reasoning: The Synergy between Pretraining and Post-Training Datacited by 27
  6. 2025RLP: Reinforcement as a Pretraining Objectivecited by 25
  7. 2025Attention Illuminates LLM Reasoning: The Preplan-and-Anchor Rhythm Enables Fine-Grained Policy Optimizationcited by 25
  8. 2025Outcome-based Reinforcement Learning to Predict the Futurecited by 22
  9. 2025Towards Concise and Adaptive Thinking in Large Reasoning Models: A Surveycited by 18
  10. 2026Entropy-Adaptive Fine-Tuning: Resolving Confident Conflicts to Mitigate Forgettingcited by 16
  11. 2026Multi-Task GRPO: Reliable LLM Reasoning Across Taskscited by 14
  12. 2025Environment Scaling for Interactive Agentic Experience Collection: A Surveycited by 13
19 more
  1. 2025K2-Think: A Parameter-Efficient Reasoning Systemcited by 11
  2. 2025Ring-lite: Scalable Reasoning via C3PO-Stabilized Reinforcement Learning for LLMscited by 10
  3. 2025RAVR: Reference-Answer-guided Variational Reasoning for Large Language Modelscited by 8
  4. 2025Reasoning Curriculum: Bootstrapping Broad LLM Reasoning from Mathcited by 8
  5. 2025Self-Improvement of Language Models by Post-Training on Multi-Agent Debatecited by 7
  6. 2025Reinforcement Learning for Tool-Integrated Interleaved Thinking towards Cross-Domain Generalizationcited by 7
  7. 2025OpenSIR: Open-Ended Self-Improving Reasonercited by 7
  8. 2026From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AIcited by 6
  9. 2025Zero Reinforcement Learning Towards General Domainscited by 5
  10. 2025K2-V2: A 360-Open, Reasoning-Enhanced LLMcited by 3
  11. 2026DeReason: A Difficulty-Aware Curriculum Improves Decoupled SFT-then-RL Training for General Reasoningcited by 3
  12. 2026How Does Reasoning Flow? Tracing Attention-Induced Information Flow for Targeted RL in LLMscited by 3
  13. 2026One Sample to Rule Them All: Extreme Data Efficiency in Multidiscipline Reasoning with Reinforcement Learningcited by 2
  14. 2026Harmony in Diversity: Multi-domain Contrastive Policy Optimization for Large Reasoning Modelscited by 2
  15. 2026Rethinking Multiple-Choice Questions for RLVR: Unlocking Potential via Distractor Designcited by 1
  16. 2026SUPERNOVA: Eliciting General Reasoning in LLMs with Reinforcement Learning on Natural Instructionscited by 0
  17. 2026CARE-RL: Capability-Aware Reinforcement Learning for Mitigating Cross-Domain Conflictscited by 0
  18. 2026Be Your Own Teacher: Steering Protein Language Models via Unsupervised Reward Optimizationcited by 0
  19. 2026Text-Centric Post-Training for Omni-Modal Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025LogicPuzzleRL: Cultivating Robust Mathematical Reasoning in LLMs via Reinforcement Learning
  2. 2025Nemotron-Math: Efficient Long-Context Distillation of Mathematical Reasoning from Multi-Mode Supervision
  3. 2025AceReason-Nemotron: Advancing Math and Code Reasoning through Reinforcement Learning
  4. 2025No Free Lunch: Rethinking Internal Feedback for LLM Reasoning
  5. 2025Beyond English-Centric Training: How Reinforcement Learning Improves Cross-Lingual Reasoning in LLMs
  6. 2025Scaling Up RL: Unlocking Diverse Reasoning in LLMs via Prolonged Training
  7. 2025Enigmata: Scaling Logical Reasoning in Large Language Models with Synthetic Verifiable Puzzles
  8. 2025QuestA: Expanding Reasoning Capacity in LLMs via Question Augmentation