# Using Common Random Numbers for Simulation-based Planning with Rollouts

arXiv: https://arxiv.org/abs/2605.04732 · HTML: https://mlsanity.com/paper/2605.04732/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 54 references that are in the map.
- First posted on arXiv 6 May 2026.

## Builds on (most cited first, 8 of 8)

- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Evolution Strategies as a Scalable Alternative to Reinforcement Learning](https://mlsanity.com/paper/1703.03864/) (2017)
- [Learning to Plan Chemical Syntheses](https://mlsanity.com/paper/1708.04202/) (2017)
- [Combining Planning and Deep Reinforcement Learning in Tactical Decision Making for Autonomous Driving](https://mlsanity.com/paper/1905.02680/) (2019)
- [PEGASUS: A Policy Search Method for Large MDPs and POMDPs](https://mlsanity.com/paper/1301.3878/) (2013)
- [Combining Neural Networks and Tree Search for Task and Motion Planning in Challenging Environments](https://mlsanity.com/paper/1703.07887/) (2017)
- [Monte-Carlo Robot Path Planning](https://mlsanity.com/paper/2208.02673/) (2022)
- [A View of the Certainty-Equivalence Method for PAC RL as an Application of the Trajectory Tree Method](https://mlsanity.com/paper/2501.02652/) (2025)

## Built on by (most cited first, 5 of 5)

- [Causal Attribution for Agentic Decisions: Estimators, Coupling, and a Traceability Specification](https://mlsanity.com/paper/2609.06445/) (2026)
- [OSCC: Certified Observation-Safe Coupling Optimization for Gradient-Noise Control in Imperfect-Information Learning](https://mlsanity.com/paper/2609.33543/) (2026)
- [VStress: Correlation-Aware Auditing and Adaptive Budget Allocation for Repeated Verifiers](https://mlsanity.com/paper/2609.36958/) (2026)
- [FSPO: Policy-Consistent Risk and Pareto-Feasible Control for Budgeted LLM RL Post-Training](https://mlsanity.com/paper/2610.02828/) (2026)
- [SafeAdapt-Poker: Counterfactual Risk Calibration for Budgeted Adaptation under Opponent Shift](https://mlsanity.com/paper/2610.20473/) (2026)

## Abstract (arXiv)

Simulation-based planning with rollouts is a widely-deployed technique for decision making in stochastic environments. The primary instrument of simulation-based planning is a sampling model, which is repeatedly called to generate trajectories and estimate the utilities of available actions. Among the actions thus explored, one with the maximum estimated utility is then executed. In this paper, we examine the effect of using common random numbers in the simulation process. We obtain a simple recipe for (provably) reducing variance in relative utility when simulations invoke a rollout policy beyond some depth. Experiments on synthetic tasks confirm that our scheme improves task performance. The broader significance of our innovation is apparent from two practical applications: (1) single-step lookahead planning in a pension-disbursement task, and (2) a deployment of the well-known UCT algorithm for the game of Ludo.