mlsanityData to 14 Oct 2026

Adaptive BESS and Grid Setpoints Optimization: A Model-Free Framework for Efficient Battery Management under Dynamic Tariff Pricing

arXiv 2408.09989 ↗First posted 19 Aug 2024eess.SY cs.SYcited by 0

On the map: Deep reinforcement learning methods → RL for energy, networks and operations

Abstract (arXiv)

This paper introduces an enhanced framework for managing Battery Energy Storage Systems (BESS) in residential communities. The non-convex BESS control problem is first addressed using a gradient-based optimizer, providing a benchmark solution. Subsequently, the problem is tackled using multiple Deep Reinforcement Learning (DRL) agents, with a specific emphasis on the off-policy Soft Actor-Critic (SAC) algorithm. This version of SAC incorporates reward refinement based on this non-convex problem, applying logarithmic scaling to enhance convergence rates. Additionally, a safety mechanism selects only feasible actions from the action space, aimed at improving the learning curve, accelerating convergence, and reducing computation times. Moreover, the state representation of this DRL approach now includes uncertainties quantified in the entropy term, enhancing the model's adaptability across various entropy types. This developed system adheres to strict limits on the battery's State of Charge (SOC), thus preventing breaches of SOC boundaries and extending the battery lifespan. The robustness of the model is validated across several Australian states' districts, each characterized by unique uncertainty distributions. By implementing the refined SAC, the SOC consistently surpasses 50 percent by the end of each day, enabling the BESS control to start smoothly for the next day with some reserve. Finally, this proposed DRL method achieves a mean reduction in optimization time by 50 percent and an average cost saving of 40 percent compared to the gradient-based optimization benchmark.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods. Builds on 12 of its 69 references that are in the map. First posted on arXiv 19 Aug 2024.

Builds on

12 of 69 references.

References · 12

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2015Continuous control with deep reinforcement learningcited by 3,407
  4. 2016Deep Learning with Differential Privacycited by 2,728
  5. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  6. 2016OpenAI Gymcited by 1,736
  7. 2016Continuous Deep Q-Learning with Model-based Accelerationcited by 157
  8. 2018A Sufficient Condition for Convergences of Adam and RMSPropcited by 126
  9. 2021Reinforcement Learning for Selective Key Applications in Power Systems: Recent Advances and Future Challengescited by 50
  10. 2016Reinforcement Learning through Asynchronous Advantage Actor-Critic on a GPUcited by 48
  11. 2021Deep Reinforcement Learning with Quantum-inspired Experience Replaycited by 19
  12. 2022Digital twin of a MWh-scale grid battery system for efficiency and degradation analysiscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Deep Reinforcement Learning-Based Optimization of Second-Life Battery Utilization in Electric Vehicles Charging Stations
  2. 2024Temporal-Aware Deep Reinforcement Learning for Energy Storage Bidding in Energy and Contingency Reserve Markets
  3. 2025Price Aware Power Split Control in Heterogeneous Battery Storage Systems ↗
  4. 2026Comparative Evaluation of MILP, MPC, and Reinforcement Learning for Commercial Battery Dispatch Under Time-of-Use Tariffs
  5. 2026Reinforcement Learning-based Home Energy Management with Heterogeneous Batteries and Stochastic EV Behaviour
  6. 2024Large Language Model Assisted Optimal Bidding of BESS in FCAS Market: An AI-agent based Approach
  7. 2025Reinforcement Learning-based Approach for Vehicle-to-Building Charging with Heterogeneous Agents and Long Term Rewards
  8. 2025Economic Battery Storage Dispatch with Deep Reinforcement Learning from Rule-Based Demonstrations