mlsanityData to 7 Oct 2026

Navigating Demand Uncertainty in Container Shipping: Deep Reinforcement Learning for Enabling Adaptive and Feasible Master Stowage Planning

arXiv 2502.12756 ↗First posted 18 Feb 2025cs.LG math.OCcited by 0

Not on the map. Why?

Abstract (arXiv)

Reinforcement learning (RL) has successfully solved various deterministic and stochastic planning problems. However, conventional RL struggles with complex real-world constraints, particularly when feasibility is explicit and depends on the current state or trajectory. In this work, we address stochastic sequential decision-making with state-dependent constraints through a real-world case study of the master stowage planning problem in container shipping, which aims to optimize revenue and costs under demand uncertainty and operational constraints. We propose a deep RL framework with an encoder-decoder model that integrates problem instance, solution, and uncertainty information to guide planning. We introduce differentiable projection layers that enforce convex polyhedral constraints, while Jacobian corrections offset the projections to yield unbiased policy gradient estimates. Experiments show that our model efficiently finds adaptive, feasible solutions that generalize across distribution shifts and scale to longer planning horizons, outperforming state-of-the-art baselines in constrained RL and stochastic programming. As such, our policies enable adaptive, uncertainty-aware planning that can support resilient and sustainable supply chains.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 32 of its 63 references that are in the map. First posted on arXiv 18 Feb 2025.

Builds on

32 of 63 references.

References · 32

  1. 2017Attention Is All You Needcited by 59,932
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2021Convex optimizationcited by 2,479
  5. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  6. 2015Pointer Networkscited by 882
  7. 2018Attention, Learn to Solve Routing Problems!cited by 646
  8. 2018Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizoncited by 612
  9. 2019Differentiable Convex Optimization Layerscited by 458
  10. 2017Safe Reinforcement Learning via Shieldingcited by 427
  11. 2020POMO: Policy Optimization with Multiple Optima for Reinforcement Learningcited by 341
  12. 2020Reinforcement Learning for Combinatorial Optimization: A Surveycited by 235
20 more
  1. 2021DC3: A learning method for optimization with hard constraintscited by 181
  2. 2021Efficient Active Search for Combinatorial Optimization Problemscited by 99
  3. 2017OptLayer - Practical Constrained Optimization for Deep Reinforcement Learning in the Real Worldcited by 64
  4. 2022Penalized Proximal Policy Optimization for Safe Reinforcement Learningcited by 60
  5. 2022Enforcing Hard Constraints with Soft Barriers: Safe Reinforcement Learning in Unknown Stochastic Environmentscited by 54
  6. 2020Action Space Shaping in Deep Reinforcement Learningcited by 45
  7. 2021State Augmented Constrained Reinforcement Learning: Overcoming the Limitations of Learning with Rewardscited by 37
  8. 2023End-to-End Feasible Optimization Proxies for Large-Scale Economic Dispatchcited by 35
  9. 2018Resource Constrained Deep Reinforcement Learningcited by 23
  10. 2022Learning to Solve Optimization Problems with Hard Linear Constraintscited by 23
  11. 2017Deep Learning Assisted Heuristic Tree Search for the Container Pre-marshalling Problemcited by 18
  12. 2021Escaping from Zero Gradient: Revisiting Action-Constrained Reinforcement Learning via Frank-Wolfe Policy Optimizationcited by 18
  13. 2018Clipped Action Policy Gradientcited by 17
  14. 2019Solving Online Threat Screening Games using Constrained Action Space Reinforcement Learningcited by 10
  15. 2023Benchmarking Actor-Critic Deep Reinforcement Learning Algorithms for Robotics Control with Action Constraintscited by 10
  16. 2017Combining Progressive Hedging with a Frank-Wolfe Method to Compute Lagrangian Dual Bounds in Stochastic Mixed-Integer Programmingcited by 2
  17. 2019On sample average approximation for two-stage stochastic programs without relatively complete recoursecited by 2
  18. 2020The multi-port berth allocation problem with speed optimization: Exact methods and a cooperative game analysiscited by 2
  19. 2023Literature Survey on the Container Stowage Planning Problemcited by 2
  20. 2024An Efficient Integer Programming Model for Solving the Master Planning Problem of Container Vessel Stowagecited by 1

Built on by

Not cited yet.