# Distributed LLM Pretraining During Renewable Curtailment Windows: A Feasibility Study

arXiv: https://arxiv.org/abs/2602.22760 · HTML: https://mlsanity.com/paper/2602.22760/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 41 references that are in the map.
- First posted on arXiv 26 Feb 2026.

## Builds on (most cited first, 14 of 14)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale](https://mlsanity.com/paper/2406.17557/) (2024)
- [Adaptive Federated Optimization](https://mlsanity.com/paper/2003.00295/) (2020)
- [Flower: A Friendly Federated Learning Research Framework](https://mlsanity.com/paper/2007.14390/) (2020)
- [Measuring the Carbon Intensity of AI in Cloud Instances](https://mlsanity.com/paper/2206.05229/) (2022)
- [Carbon Explorer: A Holistic Approach for Designing Carbon Aware Datacenters](https://mlsanity.com/paper/2201.10036/) (2022)
- [CarbonScaler: Leveraging Cloud Workload Elasticity for Optimizing Carbon-Efficiency](https://mlsanity.com/paper/2302.08681/) (2023)
- [Green Federated Learning: A new era of Green Aware AI](https://mlsanity.com/paper/2409.12626/) (2024)
- [FedZero: Leveraging Renewable Excess Energy in Federated Learning](https://mlsanity.com/paper/2305.15092/) (2023)
- [The Online Pause and Resume Problem: Optimal Algorithms and An Application to Carbon-Aware Load Shifting](https://mlsanity.com/paper/2303.17551/) (2023)
- [Accelerating Geo-distributed Machine Learning with Network-Aware Adaptive Tree and Auxiliary Route](https://mlsanity.com/paper/2404.11352/) (2024)
- [Unlocking the Potential of Renewable Energy Through Curtailment Prediction](https://mlsanity.com/paper/2405.18526/) (2024)
- [ElectricityEmissions.jl: A Framework for the Comparison of Carbon Intensity Signals](https://mlsanity.com/paper/2411.06560/) (2024)
- [Vessim: A Testbed for Carbon-Aware Applications and Systems](https://mlsanity.com/paper/2306.09774/) (2023)

## Abstract (arXiv)

Training large language models (LLMs) requires substantial compute and energy. At the same time, renewable energy sources regularly produce more electricity than the grid can absorb, leading to curtailment, the deliberate reduction of clean generation that would otherwise go to waste. These periods represent an opportunity: if training is aligned with curtailment windows, LLMs can be pretrained using electricity that is both clean and cheap. This technical report presents a system that performs full-parameter LLM training across geo-distributed GPU clusters during regional curtailment windows, elastically switching between local single-site training and federated multi-site synchronization as sites become available or unavailable. Our prototype trains a 561M-parameter transformer model across three clusters using the Flower federated learning framework, with curtailment periods derived from real-world marginal carbon intensity traces. Preliminary results show that curtailment-aware scheduling preserves training quality while reducing operational emissions to 5-12% of single-site baselines.