StartLux-Decision: AutoThink Decision Models with Reinforcement Learning for Calibrated Deliberative Decisions
On the map: Deep reinforcement learning methods → RL training of language and agent models also fits: Interpretable and logic-guided RL
Abstract (arXiv)
We introduce StartLux-Decision, decision models from 0.8B to 27B parameters that combine efficient single-pass prediction with adaptive, confidence-guided reasoning. The Instant models use bidirectional attention in full-attention layers to improve decision accuracy while keeping single-pass inference. The AutoThink models invoke explicit reasoning for uncertain decisions and answer confident cases directly. A three-stage training pipeline establishes shared decision capabilities, specializes the two variants, and equips AutoThink with reasoning through reasoning supervision and reinforcement learning. We introduce Reinforcement Learning for Calibrated Deliberative Decisions (RLCD2), which uses a frozen model and proper scoring rules to reward improvements in decision probability quality and applies a soft penalty to overlong reasoning traces. RLCD2 raises the share of reasoning traces that terminate within the generation limit by up to 48 percentage points and cuts average reasoning length by up to 65%, while largely preserving decision accuracy. On the public subset of Decision Index 0.3, the Instant models outperform the strongest evaluated comparable-scale open-weight decision models by 7.4-16.9 points, and the 27B AutoThink model reaches 73.4. We also introduce DecisionPivot, a programmatically verifiable benchmark of computational decisions built from contrast pairs that differ in a single answer-changing detail. On DecisionPivot, AutoThink outperforms its Instant counterparts by 11.5-18.4 points in pair accuracy and leads among evaluated systems. The models run on vLLM, llama.cpp and MLX, with quantized variants for lower memory use.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL training of language and agent models, one of 15 directions in Deep reinforcement learning methods. Builds on 17 of its 23 references that are in the map. First posted on arXiv 12 Oct 2026.
Builds on
17 of 23 references.
References · 17
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
- 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
- 2024A Survey on LLM-as-a-Judgecited by 1,423
- 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,345
- 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
- 2024LLM2Vec: Large Language Models Are Secretly Powerful Text Encoderscited by 404
- 2024Language Model Cascades: Token-level uncertainty and beyondcited by 121
- 2025SQL-R1: Training Natural Language to SQL Reasoning Model By Reinforcement Learningcited by 68
- 2026Evaluating and Benchmarking the System One Model Jevcited by 9
5 more
- 2026Beyond Calibration: Do a Typed-Decision Model's Probabilities Obey the Probability Axioms?cited by 6
- 2025Effective Learning for Small Reasoning Models: An Empirical Study on 0.5B Reasoning LLMscited by 3
- 2026Typed Decision Models: An Early Evidence Audit and Evaluation Checklistcited by 3
- 2026Jev thinks "I don't know'', but doesn't say it: Introducing Sys1Cal-v1 Dataset for Probability Calibrationcited by 3
- 2026System Switch: When Should a Fast Decision Model Stop and Think?cited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Learning When to Think: Shaping Adaptive Reasoning in R1-Style Models via Multi-Stage RL
- 2025ConciseRL: Conciseness-Guided Reinforcement Learning for Efficient Reasoning Models
- 2026Towards a Mechanistic Understanding of Large Reasoning Models: A Survey of Training, Inference, and Failures
- 2026OpenJev-RLCD: A Working RLCD Implementation
- 2025Learning to Reason Efficiently with Discounted Reinforcement Learning
- 2025The Markovian Thinker: Architecture-Agnostic Linear Scaling of Reasoning
- 2025RoiRL: Efficient, Self-Supervised Reasoning with Offline Iterative Reinforcement Learning
- 2025Maximizing Confidence Alone Improves Reasoning