# A Prescriptive Dirichlet Power Allocation Policy with Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2201.08445 · HTML: https://mlsanity.com/paper/2201.08445/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 77 references that are in the map.
- First posted on arXiv 20 Jan 2022.

## Builds on (most cited first, 18 of 18)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Reinforcement Learning with Deep Energy-Based Policies](https://mlsanity.com/paper/1702.08165/) (2017)
- [A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem](https://mlsanity.com/paper/1706.10059/) (2017)
- [Control of a Quadrotor with Reinforcement Learning](https://mlsanity.com/paper/1707.05110/) (2017)
- [Potential, Challenges and Future Directions for Deep Learning in Prognostics and Health Management Applications](https://mlsanity.com/paper/2005.02144/) (2020)
- [Dirichlet Variational Autoencoder](https://mlsanity.com/paper/1901.02739/) (2019)
- [Fusing Physics-based and Deep Learning Models for Prognostics](https://mlsanity.com/paper/2003.00732/) (2020)
- [Setting up a Reinforcement Learning Task with a Real-World Robot](https://mlsanity.com/paper/1803.07067/) (2018)
- [Off-Policy Reinforcement Learning for Efficient and Effective GAN Architecture Search](https://mlsanity.com/paper/2007.09180/) (2020)
- [Real-Time Model Calibration with Deep Reinforcement Learning](https://mlsanity.com/paper/2006.04001/) (2020)
- [Battery Asset Management with Cycle Life Prognosis](https://mlsanity.com/paper/2011.14903/) (2020)

## Built on by (most cited first, 4 of 4)

- [A Comprehensive Review of Digital Twin -- Part 1: Modeling and Twinning Enabling Technologies](https://mlsanity.com/paper/2208.14197/) (2022)
- [Reinforcement Learning for Topic Models](https://mlsanity.com/paper/2305.04843/) (2023)
- [Autoregressive Policy Optimization for Constrained Allocation Tasks](https://mlsanity.com/paper/2409.18735/) (2024)
- [Deep Reinforcement Learning for Adaptive Power Allocation in ISAC Systems with Mobile Target](https://mlsanity.com/paper/2606.12078/) (2026)

## Abstract (arXiv)

Prescribing optimal operation based on the condition of the system and, thereby, potentially prolonging the remaining useful lifetime has a large potential for actively managing the availability, maintenance and costs of complex systems. Reinforcement learning (RL) algorithms are particularly suitable for this type of problems given their learning capabilities. A special case of a prescriptive operation is the power allocation task, which can be considered as a sequential allocation problem, where the action space is bounded by a simplex constraint. A general continuous action-space solution of such sequential allocation problems has still remained an open research question for RL algorithms. In continuous action-space, the standard Gaussian policy applied in reinforcement learning does not support simplex constraints, while the Gaussian-softmax policy introduces a bias during training. In this work, we propose the Dirichlet policy for continuous allocation tasks and analyze the bias and variance of its policy gradients. We demonstrate that the Dirichlet policy is bias-free and provides significantly faster convergence, better performance and better hyperparameters robustness over the Gaussian-softmax policy. Moreover, we demonstrate the applicability of the proposed algorithm on a prescriptive operation case, where we propose the Dirichlet power allocation policy and evaluate the performance on a case study of a set of multiple lithium-ion (Li-I) battery systems. The experimental results show the potential to prescribe optimal operation, improve the efficiency and sustainability of multi-power source systems.