# Skip Training for Multi-Agent Reinforcement Learning Controller for Industrial Wave Energy Converters

arXiv: https://arxiv.org/abs/2209.05656 · HTML: https://mlsanity.com/paper/2209.05656/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 28 references that are in the map.
- First posted on arXiv 13 Sep 2022.

## Builds on (most cited first, 9 of 9)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games](https://mlsanity.com/paper/2103.01955/) (2021)
- [Model-Based Reinforcement Learning for Atari](https://mlsanity.com/paper/1903.00374/) (2019)
- [Benchmarking Deep Reinforcement Learning for Continuous Control](https://mlsanity.com/paper/1604.06778/) (2016)

## Built on by (most cited first, 9 of 9)

- [RTDK-BO: High Dimensional Bayesian Optimization with Reinforced Transformer Deep kernels](https://mlsanity.com/paper/2310.03912/) (2023)
- [N-Critics: Self-Refinement of Large Language Models with Ensemble of Critics](https://mlsanity.com/paper/2310.18679/) (2023)
- [Benchmark Generation Framework with Customizable Distortions for Image Classifier Robustness](https://mlsanity.com/paper/2310.18626/) (2023)
- [PyDCM: Custom Data Center Models with Reinforcement Learning for Sustainability](https://mlsanity.com/paper/2310.03906/) (2023)
- [Carbon Footprint Reduction for Sustainable Data Centers in Real-Time](https://mlsanity.com/paper/2403.14092/) (2024)
- [Robustness and Visual Explanation for Black Box Image, Video, and ECG Signal Classification with Reinforcement Learning](https://mlsanity.com/paper/2403.18985/) (2024)
- [Function Approximation for Reinforcement Learning Controller for Energy from Spread Waves](https://mlsanity.com/paper/2404.10991/) (2024)
- [A Configurable Pythonic Data Center Model for Sustainable Cooling and ML Integration](https://mlsanity.com/paper/2404.12498/) (2024)
- [Sustainability of Data Center Digital Twins with Reinforcement Learning](https://mlsanity.com/paper/2404.10786/) (2024)

## Abstract (arXiv)

Recent Wave Energy Converters (WEC) are equipped with multiple legs and generators to maximize energy generation. Traditional controllers have shown limitations to capture complex wave patterns and the controllers must efficiently maximize the energy capture. This paper introduces a Multi-Agent Reinforcement Learning controller (MARL), which outperforms the traditionally used spring damper controller. Our initial studies show that the complex nature of problems makes it hard for training to converge. Hence, we propose a novel skip training approach which enables the MARL training to overcome performance saturation and converge to more optimum controllers compared to default MARL training, boosting power generation. We also present another novel hybrid training initialization (STHTI) approach, where the individual agents of the MARL controllers can be initially trained against the baseline Spring Damper (SD) controller individually and then be trained one agent at a time or all together in future iterations to accelerate convergence. We achieved double-digit gains in energy efficiency over the baseline Spring Damper controller with the proposed MARL controllers using the Asynchronous Advantage Actor-Critic (A3C) algorithm.