mlsanityData to 14 Oct 2026

Toward Sustainable GenAI using Generation Directives for Carbon-Friendly Large Language Model Inference

arXiv 2403.12900 ↗First posted 19 Mar 2024cs.DC cs.AI cs.CLcited by 13

Not on the map. Why?

Abstract (arXiv)

The rapid advancement of Generative Artificial Intelligence (GenAI) across diverse sectors raises significant environmental concerns, notably the carbon emissions from their cloud and high performance computing (HPC) infrastructure. This paper presents Sprout, an innovative framework designed to address these concerns by reducing the carbon footprint of generative Large Language Model (LLM) inference services. Sprout leverages the innovative concept of "generation directives" to guide the autoregressive generation process, thereby enhancing carbon efficiency. Our proposed method meticulously balances the need for ecological sustainability with the demand for high-quality generation outcomes. Employing a directive optimizer for the strategic assignment of generation directives to user prompts and an original offline quality evaluator, Sprout demonstrates a significant reduction in carbon emissions by over 40% in real-world evaluations using the Llama2 LLM and global electricity grid data. This research marks a critical step toward aligning AI technology with sustainable practices, highlighting the potential for mitigating environmental impacts in the rapidly expanding domain of generative artificial intelligence.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 43 of its 74 references that are in the map. First posted on arXiv 19 Mar 2024.

Builds on

43 of 74 references.

References · 43

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  6. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  7. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  8. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  9. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  10. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
  11. 2024Mixtral of Expertscited by 1,823
  12. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
31 more
  1. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,254
  2. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
  3. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 1,006
  4. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 748
  5. 2019Green AIcited by 627
  6. 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 575
  7. 2021Carbon Emissions and Large Neural Network Trainingcited by 567
  8. 2022DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scalecited by 407
  9. 2023Prompt Cache: Modular Attention Reuse for Low-Latency Inferencecited by 268
  10. 2022Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Modelcited by 265
  11. 2021ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learningcited by 256
  12. 2023Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Timecited by 252
  13. 2021Sustainable AI: Environmental Implications, Challenges and Opportunitiescited by 242
  14. 2016Clipper: A Low-Latency Online Prediction Serving Systemcited by 191
  15. 2022The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrinkcited by 184
  16. 2023Benchmarking Foundation Models with Language-Model-as-an-Examinercited by 172
  17. 2022Measuring the Carbon Intensity of AI in Cloud Instancescited by 111
  18. 2022Accelerating Distributed MoE Training and Inference with Linacited by 109
  19. 2015Parallelizing the dual revised simplex methodcited by 89
  20. 2023SpotServe: Serving Generative Large Language Models on Preemptible Instancescited by 87
  21. 2023Fairness in Serving Large Language Modelscited by 78
  22. 2020Chasing Carbon: The Elusive Environmental Footprint of Computingcited by 72
  23. 2023LLMCarbon: Modeling the end-to-end Carbon Footprint of Large Language Modelscited by 72
  24. 2022Carbon Explorer: A Holistic Approach for Designing Carbon Aware Datacenterscited by 32
  25. 2023PIT: Optimization of Dynamic Sparse Deep Learning Models via Permutation Invariant Transformationcited by 25
  26. 2023Clover: Toward Sustainable AI with Carbon-Aware Machine Learning Inference Servicecited by 17
  27. 2022Treehouse: A Case For Carbon-Aware Datacenter Softwarecited by 12
  28. 2023Toward Sustainable HPC: Carbon Footprint Estimation and Environmental Implications of HPC Systemscited by 12
  29. 2019ALERT: Accurate Learning for Energy and Timelinesscited by 9
  30. 2022Ecovisor: A Virtual Energy System for Carbon-Efficient Applicationscited by 7
  31. 2022KAIROS: Building Cost-Efficient Machine Learning Inference Systems with Heterogeneous Cloud Resourcescited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 4; 2026: 4.

13 papers.

Sort:

By area:

  1. 2024LLM Inference Serving: Survey of Recent Advances and Opportunitiescited by 46
  2. 2024Offline Energy-Optimal LLM Serving: Workload-Based Energy Models for LLM Inference on Heterogeneous Systemscited by 28
  3. 2025Towards Sustainable NLP: Insights from Benchmarking Inference Energy in Large Language Modelscited by 11
  4. 2025Brevity is the soul of sustainability: Characterizing LLM response lengthscited by 9
  5. 2024CE-NAS: An End-to-End Carbon-Efficient Neural Architecture Search Frameworkcited by 5
  6. 2026Beyond Test-Time Compute Strategies: Advocating Energy-per-Token in LLM Inferencecited by 5
  7. 2024Impact of Model Size on Fine-tuned LLM Performance in Data-to-Text Generation: A State-of-the-Art Investigationcited by 1
  8. 2024Double Jeopardy and Climate Impact in the Use of Large Language Models: Socio-economic Disparities and Reduced Utility for Non-English Speakerscited by 1
  9. 2025How Do Companies Manage the Environmental Sustainability of AI? An Interview Study About Green AI Efforts and Regulationscited by 1
  10. 2025AOLO: Analysis and Optimization For Low-Carbon Oriented Wireless Large Language Model Servicescited by 0
  11. 2026Responsible AI for General-Purpose Systems: Overview, Challenges, and A Path Forwardcited by 0
  12. 2026Carbon-Taxed Transformers: A Green Compression Pipeline for Overgrown Language Modelscited by 0
1 more
  1. 2026From Caveman to Expert Analyst: Energy Consumption of Variable LLM Taskscited by 0