# How Green are Neural Language Models? Analyzing Energy Consumption in Text Summarization Fine-tuning

arXiv: https://arxiv.org/abs/2501.15398 · HTML: https://mlsanity.com/paper/2501.15398/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 26 references that are in the map.
- First posted on arXiv 26 Jan 2025.

## Builds on (most cited first, 17 of 17)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [SummEval: Re-evaluating Summarization Evaluation](https://mlsanity.com/paper/2007.12626/) (2020)
- [Green AI](https://mlsanity.com/paper/1907.10597/) (2019)
- [Carbon Emissions and Large Neural Network Training](https://mlsanity.com/paper/2104.10350/) (2021)
- [A Systematic Review of Green AI](https://mlsanity.com/paper/2301.11047/) (2023)
- [Growth rates of modern science: A latent piecewise growth curve approach to model publication numbers from established and new literature databases](https://mlsanity.com/paper/2012.07675/) (2020)
- [How to estimate carbon footprint when training deep learning models? A guide and review](https://mlsanity.com/paper/2306.08323/) (2023)
- [A Supervised Approach to Extractive Summarisation of Scientific Papers](https://mlsanity.com/paper/1706.03946/) (2017)
- [Generation of Highlights from Research Papers Using Pointer-Generator Networks and SciBERT Embeddings](https://mlsanity.com/paper/2302.07729/) (2023)
- [Named Entity Recognition Based Automatic Generation of Research Highlights](https://mlsanity.com/paper/2303.12795/) (2023)
- [An Analysis of Abstractive Text Summarization Using Pre-trained Models](https://mlsanity.com/paper/2303.12796/) (2023)
- [Can pre-trained language models generate titles for research papers?](https://mlsanity.com/paper/2409.14602/) (2024)

## Built on by (most cited first, 2 of 2)

- [A Phased Development Framework Enabling Islanded Operation of Sustainable AI Data Centers With Onsite Grid-Following and Grid-Forming Energy Architectures](https://mlsanity.com/paper/2607.17391/) (2026)
- [F2Agent: Financial Fusion of Agentic Intelligence for Multimodal Trading](https://mlsanity.com/paper/2608.05668/) (2026)

## Abstract (arXiv)

Artificial intelligence systems significantly impact the environment, particularly in natural language processing (NLP) tasks. These tasks often require extensive computational resources to train deep neural networks, including large-scale language models containing billions of parameters. This study analyzes the trade-offs between energy consumption and performance across three neural language models: two pre-trained models (T5-base and BART-base), and one large language model (LLaMA-3-8B). These models were fine-tuned for the text summarization task, focusing on generating research paper highlights that encapsulate the core themes of each paper. The carbon footprint associated with fine-tuning each model was measured, offering a comprehensive assessment of their environmental impact. It is observed that LLaMA-3-8B produces the largest carbon footprint among the three models. A wide range of evaluation metrics, including ROUGE, METEOR, MoverScore, BERTScore, and SciBERTScore, were employed to assess the performance of the models on the given task. This research underscores the importance of incorporating environmental considerations into the design and implementation of neural language models and calls for the advancement of energy-efficient AI methodologies.