mlsanityData to 14 Oct 2026

SAT-RL: Training Language Models for Logical Reasoning with Symbolic Solvers

arXiv 2610.14402 ↗First posted 9 Oct 2026cs.AI cs.SCcited by 0

On the map: LLM reasoning → Reinforcement learning for reasoning also fits: Mathematical reasoning

Abstract (arXiv)

Large language models (LLMs) have achieved strong performance in mathematical and logical reasoning, yet they still suffer from the curse of complexity and struggle as problem difficulty increases with the number of constraint interactions and the size of the search space. In contrast, satisfiability solvers such as Z3 provide scalable and verifiable reasoning through constraint solving under formal theories, but lack the flexibility to interpret natural-language problem descriptions. In this work, we propose SAT-RL, a reinforcement-learning (RL) framework that trains LLMs to interact with symbolic solvers through multi-turn tool execution. SAT-RL leverages synthetic data generation with verifiable solutions, supervised warm-up on solver-augmented traces, and solver-grounded reinforcement learning with tool-aware rewards. Experiments show that SAT-RL substantially improves symbolic reasoning with tools on both in-domain and out-of-domain logic tasks. Our code is available on https://github.com/sunjia72/SAT-RL.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 29 of its 61 references that are in the map. First posted on arXiv 9 Oct 2026.

Builds on

29 of 61 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025On-Policy RL with Optimal Reward Baseline29 May
  2. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Model29 May
  4. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 25

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  3. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 716
  4. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 557
  5. 2025ReTool: Reinforcement Learning for Strategic Tool Use in LLMscited by 392
  6. 2023Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoningcited by 391
  7. 2025DeepSeek-Prover-V2: Advancing Formal Mathematical Reasoning via Reinforcement Learning for Subgoal Decompositioncited by 287
  8. 2020Transformers as Soft Reasoners over Languagecited by 270
  9. 2024Self-Play Preference Optimization for Language Model Alignmentcited by 232
  10. 2020ReClor: A Reading Comprehension Dataset Requiring Logical Reasoningcited by 217
  11. 2025ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoningcited by 201
  12. 2019CLUTRR: A Diagnostic Benchmark for Inductive Reasoning from Textcited by 162
13 more
  1. 2023SatLM: Satisfiability-Aided Language Models Using Declarative Promptingcited by 102
  2. 2023NPHardEval: Dynamic Benchmark on Reasoning Ability of Large Language Models via Complexity Classescited by 79
  3. 2024CFBench: A Comprehensive Constraints-Following Benchmark for LLMscited by 67
  4. 2025Enigmata: Scaling Logical Reasoning in Large Language Models with Synthetic Verifiable Puzzlescited by 56
  5. 2021AR-LSAT: Investigating Analytical Reasoning of Textcited by 51
  6. 2023BoardgameQA: A Dataset for Natural Language Reasoning with Contradictory Informationcited by 44
  7. 2025The Choice of Divergence: A Neglected Key to Mitigating Diversity Collapse in Reinforcement Learning with Verifiable Rewardcited by 37
  8. 2025SATBench: Benchmarking LLMs' Logical Reasoning via Automated Puzzle Generation from SAT Formulascited by 33
  9. 2025AutoLogi: Automated Generation of Logic Puzzles for Evaluating Reasoning Abilities of Large Language Modelscited by 31
  10. 2025Large Language Models Meet Symbolic Provers for Logical Reasoning Evaluationcited by 28
  11. 2024P-FOLIO: Evaluating and Improving Logical Reasoning with Abundant Human-Written Reasoning Chainscited by 15
  12. 2024MCP-Solver: Integrating Language Models with Constraint Programming Systemscited by 10
  13. 2025HardcoreLogic: Challenging Large Reasoning Models with Long-tail Logic Puzzle Gamescited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning
  2. 2025A Survey of Reinforcement Learning for Large Reasoning Models
  3. 2025SATURN: SAT-based Reinforcement Learning to Unleash LLMs Reasoning
  4. 2025Reasoning Core: A Scalable RL Environment for LLM Symbolic Reasoning
  5. 2024RLSF: Fine-tuning LLMs via Symbolic Feedback
  6. 2026Can RL Teach Long-Horizon Reasoning to LLMs? Expressiveness Is Key
  7. 2025SATQuest: A Verifier for Logical Reasoning Evaluation and Reinforcement Fine-Tuning of LLMs
  8. 2025Scaling Up RL: Unlocking Diverse Reasoning in LLMs via Prolonged Training