# Toward an Energy-Optimized Operation of Data Centers Located in Wind Farms Using Reinforcement Learning

arXiv: https://arxiv.org/abs/2606.30316 · HTML: https://mlsanity.com/paper/2606.30316/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Decision transformers and sequence modeling, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 9 of its 34 references that are in the map.
- First posted on arXiv 29 Jun 2026.

## Placed in

- Offline RL and imitation learning from data / [Decision transformers and sequence modeling](https://mlsanity.com/offlinerl/decision-transformers-and-sequence-modeling/)

## Builds on (most cited first, 9 of 9)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Carbon-Aware Computing for Datacenters](https://mlsanity.com/paper/2106.11750/) (2021)
- [Guided Exploration with Proximal Policy Optimization using a Single Demonstration](https://mlsanity.com/paper/2007.03328/) (2020)
- [On the Limitations of Carbon-Aware Temporal and Spatial Workload Shifting in the Cloud](https://mlsanity.com/paper/2306.06502/) (2023)
- [Adapting Datacenter Capacity for Greener Datacenters and Grid](https://mlsanity.com/paper/2301.03148/) (2023)
- [A Reinforcement Learning Based Backfilling Strategy for HPC Batch Jobs](https://mlsanity.com/paper/2404.09264/) (2024)
- [A Framework for Carbon-aware Real-Time Workload Management in Clouds using Renewables-driven Cores](https://mlsanity.com/paper/2411.07628/) (2024)

## Abstract (arXiv)

This paper studies Reinforcement Learning as an online controller for curtailment-aware workload shifting in wind-turbine-integrated high-performance computing (HPC) data centers. We introduce a reproducible fixed-day simulation framework with synthetic wind and price signals and delayed completion feedback, designed to be extensible toward more complex scenarios. As a controlled benchmarking basis, we then focus on the minimal case with one wind turbine and one co-located data center. In this setting, pure Reinforcement Learning exhibits a pronounced credit-assignment problem and tends to underuse free wind energy early in the day. We therefore evaluate two complementary countermeasures: optimization-based Imitation Learning and potential-based Reward Shaping. Across multi-seed training and a 200-day test set, Proximal Policy Optimization (PPO) and a Soft Actor-Critic (SAC) variant with an additional on-policy update routine achieve strong empirical performance among learned policies, and both Imitation Learning and Reward Shaping provide improvements in relevant configurations. A performance gap to the optimizer remains, which is expected: the optimizer plans offline with full-day foresight, whereas Reinforcement Learning must decide online from current observations without future realizations. The benchmark and ablation results provide a transparent basis for extending the approach toward richer multi-site and continuous-time scenarios.