mlsanityData to 14 Oct 2026

Function Approximation for Reinforcement Learning Controller for Energy from Spread Waves

arXiv 2404.10991 ↗First posted 17 Apr 2024cs.AI cs.LG cs.SYcited by 1

On the map: Deep reinforcement learning methods → RL for energy, networks and operations

Abstract (arXiv)

The industrial multi-generator Wave Energy Converters (WEC) must handle multiple simultaneous waves coming from different directions called spread waves. These complex devices in challenging circumstances need controllers with multiple objectives of energy capture efficiency, reduction of structural stress to limit maintenance, and proactive protection against high waves. The Multi-Agent Reinforcement Learning (MARL) controller trained with the Proximal Policy Optimization (PPO) algorithm can handle these complexities. In this paper, we explore different function approximations for the policy and critic networks in modeling the sequential nature of the system dynamics and find that they are key to better performance. We investigated the performance of a fully connected neural network (FCN), LSTM, and Transformer model variants with varying depths and gated residual connections. Our results show that the transformer model of moderate depth with gated residual connections around the multi-head attention, multi-layer perceptron, and the transformer block (STrXL) proposed in this paper is optimal and boosts energy efficiency by an average of 22.1% for these complex spread waves over the existing spring damper (SD) controller. Furthermore, unlike the default SD controller, the transformer controller almost eliminated the mechanical stress from the rotational yaw motion for angled waves. Demo: https://tinyurl.com/yueda3jh

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods. Builds on 14 of its 46 references that are in the map. First posted on arXiv 17 Apr 2024.

Builds on

14 of 46 references.

References · 14

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  5. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  6. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  7. 2019Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecastingcited by 761
  8. 2016Benchmarking Deep Reinforcement Learning for Continuous Controlcited by 371
  9. 2019Stabilizing Transformers for Reinforcement Learningcited by 270
  10. 2017Parameter Space Noise for Explorationcited by 161
  11. 2021State Entropy Maximization with Random Encoders for Efficient Explorationcited by 108
  12. 2023RTDK-BO: High Dimensional Bayesian Optimization with Reinforced Transformer Deep kernelscited by 11
2 more
  1. 2022Skip Training for Multi-Agent Reinforcement Learning Controller for Industrial Wave Energy Converterscited by 9
  2. 2023Benchmark Generation Framework with Customizable Distortions for Image Classifier Robustnesscited by 7

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024SustainDC: Benchmarking for Sustainable Data Center Controlcited by 8

Similar, not cited

Close in wording, no citation either way

  1. 2026Control of hybrid wind-wave energy systems using reinforcement learning ↗
  2. 2026Coupling Smoothed Particle Hydrodynamics with Multi-Agent Deep Reinforcement Learning for Cooperative Control of Point Absorbers ↗
  3. 2024Deep Reinforcement Learning for Optimizing Inverter Control: Fixed and Adaptive Gain Tuning Strategies for Power System Stability
  4. 2025Optimizing Power Grid Topologies with Reinforcement Learning: A Survey of Methods and Challenges
  5. 2025Wide-Area Feedback Control for Renewables-Heavy Power Systems: A Comparative Study of Reinforcement Learning and Lyapunov-Based Design
  6. 2024Multi-Agent Reinforcement Learning for Energy Networks: Computational Challenges, Progress and Open Problems
  7. 2025Reinforcement Learning-Based Controlled Switching Approach for Inrush Current Minimization in Power Transformers
  8. 2024Safety Constrained Multi-Agent Reinforcement Learning for Active Voltage Control