# AI Greenferencing: Routing AI Inferencing to Green Modular Data Centers with Heron

arXiv: https://arxiv.org/abs/2505.09989 · HTML: https://mlsanity.com/paper/2505.09989/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 67 references that are in the map.
- First posted on arXiv 15 May 2025.

## Builds on (most cited first, 9 of 9)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting](https://mlsanity.com/paper/1912.09363/) (2019)
- [DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving](https://mlsanity.com/paper/2401.09670/) (2024)
- [A Study of Generative Large Language Model for Medical Research and Healthcare](https://mlsanity.com/paper/2305.13523/) (2023)
- [Exploring the Efficiency of Renewable Energy-based Modular Data Centers at Scale](https://mlsanity.com/paper/2406.02252/) (2024)
- [Dissecting Latency in the Internet's Fiber Infrastructure](https://mlsanity.com/paper/1811.10737/) (2018)

## Built on by (most cited first, 3 of 3)

- [CarbonX: An Open-Source Tool for Computational Decarbonization Using Time Series Foundation Models](https://mlsanity.com/paper/2510.01521/) (2025)
- [VoltanaLLM: Energy-Efficient and SLO-Aware Disaggregated LLM Serving via Adaptive Frequency Control and State-Space Routing](https://mlsanity.com/paper/2509.04827/) (2025)
- [Enabling Spatially Fine-Grained DVFS in Neural Processing Units for Energy-Efficient LLM Serving](https://mlsanity.com/paper/2607.16473/) (2026)

## Abstract (arXiv)

AI power demand is growing unprecedentedly thanks to the high power density of AI compute and the emerging inferencing workload. On the supply side, abundant wind power is waiting for grid access in interconnection queues. In this light, this paper argues bringing AI workload to modular compute clusters co-located in wind farms. Our deployment right-sizing strategy makes it economically viable to deploy more than 6 million high-end GPUs today that could consume cheap, green power at its source. We built Heron, a cross-site software router, that could efficiently leverage the complementarity of power generation across wind farms by routing AI inferencing workload around power drops. Using 1-week ofcoding and conversation production traces from Azure and (real) variable wind power traces, we show how Heron improves aggregate goodput of AI compute by up to 80% compared to the state-of-the-art.