mlsanityData to 14 Oct 2026

Reasoning Core: A Scalable RL Environment for LLM Symbolic Reasoning

arXiv 2509.18083 ↗First posted 22 Sep 2025cs.AI cs.CLcited by 5

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

We introduce Reasoning Core, a new scalable environment for Reinforcement Learning with Verifiable Rewards (RLVR), designed to advance foundational symbolic reasoning in Large Language Models (LLMs). Unlike existing benchmarks that focus on games or isolated puzzles, Reasoning Core procedurally generates problems across core formal domains, including PDDL planning, first-order logic, context-free grammar parsing, causal reasoning, and system equation solving. The environment is built on key design principles of high-generality problem distributions, verification via external tools, and continuous difficulty control, which together provide a virtually infinite supply of novel training instances. Initial zero-shot evaluations with frontier LLMs confirm the difficulty of Reasoning Core's tasks, positioning it as a promising resource to improve the reasoning capabilities of future models.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 20 of its 27 references that are in the map. First posted on arXiv 22 Sep 2025.

Builds on

20 of 27 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Data6 May
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan
  3. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Training22 Nov

Other references · 17

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  4. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
  5. 2024OpenAI o1 System Cardcited by 1,244
  6. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 483
  7. 2021Dynabench: Rethinking Benchmarking in NLPcited by 459
  8. 2025ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoningcited by 201
  9. 2022Will we run out of data? Limits of LLM scaling based on human-generated datacited by 195
  10. 2025ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systemscited by 190
  11. 2025REASONING GYM: Reasoning Environments for Reinforcement Learning with Verifiable Rewardscited by 130
  12. 2025The Leaderboard Illusioncited by 66
5 more
  1. 2025AutoLogi: Automated Generation of Logic Puzzles for Evaluating Reasoning Abilities of Large Language Modelscited by 31
  2. 2025Sudoku-Bench: Evaluating creative reasoning with Sudoku variantscited by 22
  3. 2025Logic Haystacks: Probing LLMs Long-Context Logical Reasoning (Without Easily Identifiable Unrelated Padding)cited by 3
  4. 2024Attention Overflow: Language Model Input Blur during Long-Context Missing Items Recommendationcited by 2
  5. 2025Saturation-Driven Dataset Generation for LLM Mathematical Reasoning in the TPTP Ecosystemcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 4.

5 papers.

  1. 2026Golden Goose: A Simple Trick to Synthesize Unlimited RLVR Tasks from Unverifiable Internet Textcited by 10
  2. 2026Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Applicationcited by 8
  3. 2026Breaking the Solver Bottleneck: Training Task Generators at the Learnable Frontiercited by 3
  4. 2025Structured In-context Environment Scaling for Large Language Model Reasoningcited by 0
  5. 2026MagicBeanStalk: Escaping the RLVR Zero Advantage Trap with Recursive Data Synthesiscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Reasoning Core: A Scalable Procedural Data Generation Suite for Symbolic Pre-training and Post-Training
  2. 2025A Survey of Reinforcement Learning for Large Reasoning Models
  3. 2025CoreThink: A Symbolic Reasoning Layer to reason over Long Horizon Tasks with LLMs
  4. 2025Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective
  5. 2026Can RL Teach Long-Horizon Reasoning to LLMs? Expressiveness Is Key
  6. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?
  7. 2025Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning
  8. 2026SAT-RL: Training Language Models for Logical Reasoning with Symbolic Solvers