mlsanityData to 14 Oct 2026

Cost-Aware Reinforcement Learning with Action Masking and Projection for Battery Energy Storage Dispatch under Suppressed-Spread Market Shifts

arXiv 2609.23590 ↗First posted 20 Sep 2026cs.LG cs.AI cs.SYcited by 0

On the map: Deep reinforcement learning methods → RL for energy, networks and operations

Abstract (arXiv)

Battery energy storage system (BESS) dispatch must preserve operational feasibility while declining price spreads reduce the margin available to pay for cycling. We study a proximal policy optimization (PPO) controller whose pre-selection physical action mask and emergency projection are separated from a causal, forecast-informed economic advisory. All forecast-dependent methods receive the same causal 24-step forecast and grid-side settlement. Across five PPO seeds, advice-on net profit is 30.59 and 18.04 USD per 336-hour T1 and T2 window, versus 36.77 and 22.94 USD for proxy-cost MPC; PPO remains below this reference in both periods. Advice raises T2 profit from 16.45 to 18.04 USD while reducing throughput, but is immaterial in T1. On disjoint weekly blocks, PPO is stable under daily, weekly, and blended seasonal forecasts, weakens under persistence, and remains below proxy-cost MPC. Paired diagnostics localize changes to the observed 5-10 USD/MWh regime with mixed SoC-dependent effects. An M0-M6 ablation shows that mask removal sends thousands of infeasible requests to projection, while removing both physical layers exposes ramp violations. The evidence separates economic screening from feasibility enforcement without claiming formal safety, lifecycle-optimal aging, or RL dominance.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods. Builds on 7 of its 19 references that are in the map. First posted on arXiv 20 Sep 2026.

Builds on

7 of 19 references.

References · 7

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2017Constrained Policy Optimizationcited by 983
  4. 2020A Closer Look at Invalid Action Masking in Policy Gradient Algorithmscited by 171
  5. 2020Safe Reinforcement Learning via Projection on a Safe Set: How to Achieve Optimality?cited by 34
  6. 2020Battery Asset Management with Cycle Life Prognosiscited by 2
  7. 2025Distributionally Robust Self Paced Curriculum Reinforcement Learningcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Comparative Evaluation of MILP, MPC, and Reinforcement Learning for Commercial Battery Dispatch Under Time-of-Use Tariffs
  2. 2025Economic Battery Storage Dispatch with Deep Reinforcement Learning from Rule-Based Demonstrations
  3. 2024Should we use model-free or model-based control? A case study of battery management systems
  4. 2024Time-Varying Constraint-Aware Reinforcement Learning for Energy Storage Control
  5. 2025Price Aware Power Split Control in Heterogeneous Battery Storage Systems ↗
  6. 2024Reinforcement learning based demand charge minimization using energy storage ↗
  7. 2025The Economic Dispatch of Power-to-Gas Systems with Deep Reinforcement Learning:Tackling the Challenge of Delayed Rewards with Long-Term Energy Storage
  8. 2026RL-PaO: Prediction as Action in Decision Making under Uncertainty