# SATURN: SAT-based Reinforcement Learning to Unleash LLMs Reasoning

arXiv: https://arxiv.org/abs/2505.16368 · HTML: https://mlsanity.com/paper/2505.16368/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning.
- Builds on 3 of its 54 references that are in the map.
- First posted on arXiv 22 May 2025.

## Placed in

- LLM reasoning / [Reinforcement learning for reasoning](https://mlsanity.com/lr/reinforcement-learning-for-reasoning/)

## Builds on (most cited first, 3 of 3)

- [Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning](https://mlsanity.com/paper/2502.14768/) (2025)
- [Qwen2.5-1M Technical Report](https://mlsanity.com/paper/2501.15383/) (2025)
- [Z1: Efficient Test-time Scaling with Code](https://mlsanity.com/paper/2504.00810/) (2025)

## Built on by (most cited first, 8 of 8)

- [SCALER:Synthetic Scalable Adaptive Learning Environment for Reasoning](https://mlsanity.com/paper/2601.04809/) (2026)
- [EvoCoT: Overcoming the Exploration Bottleneck in Reinforcement Learning](https://mlsanity.com/paper/2508.07809/) (2025)
- [Detecting Data Contamination from Reinforcement Learning Post-training for Large Language Models](https://mlsanity.com/paper/2510.09259/) (2025)
- [Breaking the Solver Bottleneck: Training Task Generators at the Learnable Frontier](https://mlsanity.com/paper/2606.18284/) (2026)
- [R^3: Replay, Reflection, and Ranking Rewards for LLM Reinforcement Learning](https://mlsanity.com/paper/2601.19620/) (2026)
- [Preventing Curriculum Collapse in Self-Evolving Reasoning Systems](https://mlsanity.com/paper/2603.13309/) (2026)
- [Can RL Teach Long-Horizon Reasoning to LLMs? Expressiveness Is Key](https://mlsanity.com/paper/2605.06638/) (2026)
- [XLGoBench: Detecting cross-lingual skill gaps with algorithmic tasks](https://mlsanity.com/paper/2605.30788/) (2026)

## Abstract (arXiv)

How to design reinforcement learning (RL) tasks that effectively unleash the reasoning capability of large language models (LLMs) remains an open question. Existing RL tasks (e.g., math, programming, and constructing reasoning tasks) suffer from three key limitations: (1) Scalability. They rely heavily on human annotation or expensive LLM synthesis to generate sufficient training data. (2) Verifiability. LLMs' outputs are hard to verify automatically and reliably. (3) Controllable Difficulty. Most tasks lack fine-grained difficulty control, making it hard to train LLMs to develop reasoning ability from easy to hard. To address these limitations, we propose Saturn, a SAT-based RL framework that uses Boolean Satisfiability (SAT) problems to train and evaluate LLMs reasoning. Saturn enables scalable task construction, rule-based verification, and precise difficulty control. Saturn designs a curriculum learning pipeline that continuously improves LLMs' reasoning capability by constructing SAT tasks of increasing difficulty and training LLMs from easy to hard. To ensure stable training, we design a principled mechanism to control difficulty transitions. We introduce Saturn-2.6k, a dataset of 2,660 SAT problems with varying difficulty. It supports the evaluation of how LLM reasoning changes with problem difficulty. We apply Saturn to DeepSeek-R1-Distill-Qwen and obtain Saturn-1.5B and Saturn-7B. We achieve several notable results: (1) On SAT problems, Saturn-1.5B and Saturn-7B achieve average pass@3 improvements of +14.0 and +28.1, respectively. (2) On math and programming tasks, Saturn-1.5B and Saturn-7B improve average scores by +4.9 and +1.8 on benchmarks (e.g., AIME, LiveCodeBench). (3) Compared to the state-of-the-art (SOTA) approach in constructing RL tasks, Saturn achieves further improvements of +8.8%. We release the source code, data, and models to support future research.