# Surpassing legacy approaches to PWR core reload optimization with single-objective Reinforcement learning

arXiv: https://arxiv.org/abs/2402.11040 · HTML: https://mlsanity.com/paper/2402.11040/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 3 of its 66 references that are in the map.
- First posted on arXiv 16 Feb 2024.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 3 of 3)

- [Reinforcement Learning for Combinatorial Optimization: A Survey](https://mlsanity.com/paper/2003.03600/) (2020)
- [Assessment of Reinforcement Learning Algorithms for Nuclear Power Plant Fuel Optimization](https://mlsanity.com/paper/2305.05812/) (2023)
- [Multi-Objective Reinforcement Learning-based Approach for Pressurized Water Reactor Optimization](https://mlsanity.com/paper/2312.10194/) (2023)

## Built on by (most cited first, 3 of 3)

- [Techno-economic optimization of a heat-pipe microreactor, part I: theory and cost optimization](https://mlsanity.com/paper/2512.16032/) (2025)
- [Techno-economic optimization of a heat-pipe microreactor, part II: multi-objective optimization analysis](https://mlsanity.com/paper/2601.20079/) (2026)
- [MAEO: Multiobjective Animorphic Ensemble Optimization for Scalable Large-scale Engineering Applications](https://mlsanity.com/paper/2604.26973/) (2026)

## Abstract (arXiv)

Optimizing the fuel cycle cost through the optimization of nuclear reactor core loading patterns involves multiple objectives and constraints, leading to a vast number of candidate solutions that cannot be explicitly solved. To advance the state-of-the-art in core reload patterns, we have developed methods based on Deep Reinforcement Learning (DRL) for both single- and multi-objective optimization. Our previous research has laid the groundwork for these approaches and demonstrated their ability to discover high-quality patterns within a reasonable time frame. On the other hand, stochastic optimization (SO) approaches are commonly used in the literature, but there is no rigorous explanation that shows which approach is better in which scenario. In this paper, we demonstrate the advantage of our RL-based approach, specifically using Proximal Policy Optimization (PPO), against the most commonly used SO-based methods: Genetic Algorithm (GA), Parallel Simulated Annealing (PSA) with mixing of states, and Tabu Search (TS), as well as an ensemble-based method, Prioritized Replay Evolutionary and Swarm Algorithm (PESA). We found that the LP scenarios derived in this paper are amenable to a global search to identify promising research directions rapidly, but then need to transition into a local search to exploit these directions efficiently and prevent getting stuck in local optima. PPO adapts its search capability via a policy with learnable weights, allowing it to function as both a global and local search method. Subsequently, we compared all algorithms against PPO in long runs, which exacerbated the differences seen in the shorter cases. Overall, the work demonstrates the statistical superiority of PPO compared to the other considered algorithms.