Reinforcement Learning with a Bilevel World-Model Architecture for Scan-Order Optimisation in Laser Directed Energy Deposition
On the map: World models → Latent dynamics models and model-based RL
Abstract (arXiv)
Scan-order design in laser directed energy deposition (LDED) is a delayed, path-dependent thermo-mechanical decision problem, because sequence quality becomes observable only after the complete deposition and cooling cycle. This work formulates LDED scan-order optimisation as a finite-horizon, permutation-constrained reinforcement-learning problem and develops a bilevel finite-element-teacher-labelled AI workflow. A surrogate-assisted teacher-guided optimisation loop learns the Abaqus-labelled response landscape and provides a tractable terminal-reward environment for policy training. A frozen Maskable Proximal Policy Optimization (MaskablePPO) policy is then used to generate legal scan-order candidates, which are independently validated through Abaqus thermo-mechanical simulations. The results show bounded, N-dependent policy-generation value rather than record-level dominance over the mature surrogate-assisted optimiser. The strongest scan orders are obtained by the teacher-guided surrogate loop, whereas PPO autonomously reaches competitive regions of the native response landscape, with stronger rank concentration at smaller track counts and a clear reliability boundary at longer horizons. The teacher-labelled landscape further supports a physically gated lexicographic reward hierarchy in which warpage admissibility is the primary constraint, plastic strain acts as a safety filter and residual-stress-related improvement is pursued conditionally within the admissible region. Validated sequences also reveal an interpretable scale-separated ordering tendency that combines global spatial dispersion with local structured grouping. This workflow provides a route from fixed scan-rule selection toward finite-element-teacher-validated policy generation, while preserving independent finite-element validation as the final physical gate.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 8 of its 24 references that are in the map. First posted on arXiv 24 May 2026.
Builds on
8 of 24 references.
References · 8
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2018Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizoncited by 612
- 2020Reinforcement Learning for Combinatorial Optimization: A Surveycited by 235
- 2020A Closer Look at Invalid Action Masking in Policy Gradient Algorithmscited by 160
- 2022Lexicographic Multi-Objective Reinforcement Learningcited by 17
- 2020Effective Reinforcement Learning through Evolutionary Surrogate-Assisted Prescriptioncited by 7
- 2024Thresholded Lexicographic Ordered Multiobjective Reinforcement Learningcited by 7
- 2024Deep Reinforcement Learning Based Toolpath Generation for Thermal Uniformity in Laser Powder Bed Fusion Processcited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2023Contrastive Energy Prediction for Exact Energy-Guided Diffusion Sampling in Offline Reinforcement Learning
- 2025Model-free Optical Processors using In Situ Reinforcement Learning with Proximal Policy Optimization
- 2026MARCO: Multi-Round Agentic Reinforcement for Conditional Molecular Optimization
- 2025Heterogeneous Adaptive Policy Optimization: Tailoring Optimization to Every Token's Nature
- 2023Revisiting Energy Based Models as Policies: Ranking Noise Contrastive Estimation and Interpolating Energy Models
- 2026ANO: Robust Policy Optimization via Bounded, Redescending Gain Fields
- 2023A dynamical clipping approach with task feedback for Proximal Policy Optimization
- 2025FlowQ: Energy-Guided Flow Policies for Offline Reinforcement Learning