mlsanityData to 14 Oct 2026

Cache Your Prompt When It's Green: Carbon-Aware Caching for Large Language Model Serving

arXiv 2505.23970 ↗First posted 29 May 2025cs.DC cs.ARcited by 1

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

As large language models (LLMs) become widely used, their environmental impact, especially carbon emission, has attracted more attention. Prior studies focus on compute-related carbon emissions. In this paper, we find that storage is another key contributor. LLM caching, which saves and reuses KV caches for repeated context, reduces operational carbon by avoiding redundant computation. However, this benefit comes at the cost of embodied carbon from high-capacity, high-speed SSDs. As LLMs scale, the embodied carbon of storage grows significantly. To address this tradeoff, we present GreenCache, a carbon-aware cache management framework that dynamically derives resource allocation plans for LLM serving. GreenCache analyzes the correlation between carbon emission and SLO satisfaction, reconfiguring the resource over time to keep the balance between SLO and carbon emission under dynamic workloads. Evaluations from real traces demonstrate that GreenCache achieves an average carbon reduction of 15.1 % when serving Llama-3 70B in the FR grid, with reductions reaching up to 25.3 %, while staying within latency constraints for > 90 % of requests.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 5 of its 9 references that are in the map. First posted on arXiv 29 May 2025.

Builds on

5 of 9 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Serving11 Oct
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttention12 Sep

Other references · 3

  1. 2024GreenLLM: Disaggregating Large Language Model Serving on Heterogeneous GPUs for Lower Carbon Emissionscited by 10
  2. 2022The Dirty Secret of SSDs: Embodied Carboncited by 6
  3. 2025EnsembleCI: Ensemble Learning for Carbon Intensity Forecastingcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Beyond Energy: When Sustainability Dimensions Reshape LLM Serving Decisionscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Towards Sustainable Large Language Model Serving
  2. 2024Harnessing Your DRAM and SSD for Sustainable and Accessible LLM Inference with Mixed-Precision and Multi-level Caching
  3. 2026Dissecting the Carbon-SLO Tension in Geo-Distributed LLM Serving
  4. 2025Strata: Hierarchical Context Caching for Long Context Language Model Serving
  5. 2025CarbonCall: Sustainability-Aware Function Calling for Large Language Models on Edge Devices
  6. 2025KVCache Cache in the Wild: Characterizing and Optimizing KVCache Cache at a Large Cloud Provider
  7. 2025FastCache: Optimizing Multimodal LLM Serving through Lightweight KV-Cache Compression Framework
  8. 2025EcoServe: Designing Carbon-Aware AI Inference Systems