# Reinforcement Learning for Efficient Design and Control Co-optimisation of Energy Systems

arXiv: https://arxiv.org/abs/2406.19825 · HTML: https://mlsanity.com/paper/2406.19825/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 12 of its 33 references that are in the map.
- First posted on arXiv 28 Jun 2024.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 12 of 12)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Sim-to-Real Transfer of Robotic Control with Dynamics Randomization](https://mlsanity.com/paper/1710.06537/) (2017)
- [An Introduction to Deep Reinforcement Learning](https://mlsanity.com/paper/1811.12560/) (2018)
- [Understanding the impact of entropy on policy optimization](https://mlsanity.com/paper/1811.11214/) (2018)
- [Reinforcement Learning for Improving Agent Design](https://mlsanity.com/paper/1810.03779/) (2018)
- [Jointly Learning to Construct and Control Agents using Deep Reinforcement Learning](https://mlsanity.com/paper/1801.01432/) (2018)
- [Evolution Gym: A Large-Scale Benchmark for Evolving Soft Robots](https://mlsanity.com/paper/2201.09863/) (2022)
- [Data-efficient Co-Adaptation of Morphology and Behaviour with Deep Reinforcement Learning](https://mlsanity.com/paper/1911.06832/) (2019)
- [Hardware as Policy: Mechanical and Computational Co-Optimization using Deep Reinforcement Learning](https://mlsanity.com/paper/2008.04460/) (2020)
- [Integrating process design and control using reinforcement learning](https://mlsanity.com/paper/2108.05242/) (2021)
- [Jointly Learning Environments and Control Policies with Projected Stochastic Gradient Ascent](https://mlsanity.com/paper/2006.01738/) (2020)
- [Reinforcement Learning for Joint Design and Control of Battery-PV Systems](https://mlsanity.com/paper/2307.04244/) (2023)

## Built on by (most cited first, 3 of 3)

- [Trustworthy and Explainable Deep Reinforcement Learning for Safe and Energy-Efficient Process Control: A Use Case in Industrial Compressed Air Systems](https://mlsanity.com/paper/2512.18317/) (2025)
- [A Reinforcement Learning-based Transmission Expansion Framework Considering Strategic Bidding in Electricity Markets](https://mlsanity.com/paper/2602.19421/) (2026)
- [Environment Parameter Gradient Theorem for Co-Design in Reinforcement Learning](https://mlsanity.com/paper/2607.12590/) (2026)

## Abstract (arXiv)

The ongoing energy transition drives the development of decentralised renewable energy sources, which are heterogeneous and weather-dependent, complicating their integration into energy systems. This study tackles this issue by introducing a novel reinforcement learning (RL) framework tailored for the co-optimisation of design and control in energy systems. Traditionally, the integration of renewable sources in the energy sector has relied on complex mathematical modelling and sequential processes. By leveraging RL's model-free capabilities, the framework eliminates the need for explicit system modelling. By optimising both control and design policies jointly, the framework enhances the integration of renewable sources and improves system efficiency. This contribution paves the way for advanced RL applications in energy management, leading to more efficient and effective use of renewable energy sources.