UltraLogic: Enhancing LLM Reasoning through Large-Scale Data Synthesis and Bipolar Float Reward
On the map: LLM reasoning → Reinforcement learning for reasoning
also in 1 more
Abstract (arXiv)
While Large Language Models (LLMs) have demonstrated significant potential in natural language processing , complex general-purpose reasoning requiring multi-step logic, planning, and verification remains a critical bottleneck. Although Reinforcement Learning with Verifiable Rewards (RLVR) has succeeded in specific domains , the field lacks large-scale, high-quality, and difficulty-calibrated data for general reasoning. To address this, we propose UltraLogic, a framework that decouples the logical core of a problem from its natural language expression through a Code-based Solving methodology to automate high-quality data production. The framework comprises hundreds of unique task types and an automated calibration pipeline across ten difficulty levels. Furthermore, to mitigate binary reward sparsity and the Non-negative Reward Trap, we introduce the Bipolar Float Reward (BFR) mechanism, utilizing graded penalties to effectively distinguish perfect responses from those with logical flaws. Our experiments demonstrate that task diversity is the primary driver for reasoning enhancement , and that BFR, combined with a difficulty matching strategy, significantly improves training efficiency, guiding models toward global logical optima.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 14 of its 27 references that are in the map. First posted on arXiv 6 Jan 2026.
Builds on
14 of 27 references.
Closest ancestors · 4
Same direction, newest first
- 2025OpenSIR: Open-Ended Self-Improving Reasoner1 Nov
- 2025Staying in the Sweet Spot: Responsive Reasoning Evolution via Capability-Adaptive Hint Scaffolding8 Sep
- 2025SynLogic: Synthesizing Verifiable Reasoning Data at Scale for Learning Logical Reasoning and Beyond26 May
- 2023Let's reward step by step: Step-Level reward model as the Navigators for Reasoning16 Oct
Other references · 10
- 2023GPT-4 Technical Reportcited by 18,132
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2025Qwen3 Technical Reportcited by 8,195
- 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
- 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 724
- 2024DeepSeek-V3 Technical Reportcited by 724
- 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
- 2025AgenticMath: Enhancing LLM Reasoning via Agentic-based Math Data Generationcited by 7
- 2025MorphoBench: A Benchmark with Difficulty Adaptive to Model Reasoningcited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025How Difficulty-Aware Staged Reinforcement Learning Enhances LLMs' Reasoning Capabilities: A Preliminary Experimental Study
- 2025Scaling Up RL: Unlocking Diverse Reasoning in LLMs via Prolonged Training
- 2025SATURN: SAT-based Reinforcement Learning to Unleash LLMs Reasoning
- 2025Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective
- 2025Revisiting LLM Reasoning via Information Bottleneck
- 2026Likelihood-Based Reward Designs for General LLM Reasoning
- 2025A Survey of Reinforcement Learning for Large Reasoning Models
- 2025Enhancing Large Language Model Reasoning with Reward Models: An Analytical Survey