# Estimating Environmental Cost Throughout Model's Adaptive Life Cycle

arXiv: https://arxiv.org/abs/2408.01446 · HTML: https://mlsanity.com/paper/2408.01446/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 41 references that are in the map.
- First posted on arXiv 23 Jul 2024.

## Builds on (most cited first, 26 of 26)

- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization](https://mlsanity.com/paper/2006.16241/) (2020)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty](https://mlsanity.com/paper/1912.02781/) (2019)
- [Carbon Emissions and Large Neural Network Training](https://mlsanity.com/paper/2104.10350/) (2021)
- [Quantifying the Carbon Emissions of Machine Learning](https://mlsanity.com/paper/1910.09700/) (2019)
- [Efficient Test-Time Model Adaptation without Forgetting](https://mlsanity.com/paper/2204.02610/) (2022)
- [Wasserstein Auto-Encoders](https://mlsanity.com/paper/1711.01558/) (2017)
- [On the Importance of Gradients for Detecting Distributional Shifts in the Wild](https://mlsanity.com/paper/2110.00218/) (2021)
- [A Fourier Perspective on Model Robustness in Computer Vision](https://mlsanity.com/paper/1906.08988/) (2019)
- [Puzzle Mix: Exploiting Saliency and Local Statistics for Optimal Mixup](https://mlsanity.com/paper/2009.06962/) (2020)
- [Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learning](https://mlsanity.com/paper/2002.05651/) (2020)
- [Generalisation in humans and deep neural networks](https://mlsanity.com/paper/1808.08750/) (2018)
- [Adversarial Examples Are a Natural Consequence of Test Error in Noise](https://mlsanity.com/paper/1901.10513/) (2019)
- [TTN: A Domain-Shift Aware Batch Normalization in Test-Time Adaptation](https://mlsanity.com/paper/2302.05155/) (2023)
- [Test-Time Adaptation via Conjugate Pseudo-labels](https://mlsanity.com/paper/2207.09640/) (2022)
- [Wavelet Integrated CNNs for Noise-Robust Image Classification](https://mlsanity.com/paper/2005.03337/) (2020)
- [Eco2AI: carbon emissions tracking of machine learning models as the first step towards sustainable AI](https://mlsanity.com/paper/2208.00406/) (2022)
- [Gradients as a Measure of Uncertainty in Neural Networks](https://mlsanity.com/paper/2008.08030/) (2020)
- [A Survey on Green Deep Learning](https://mlsanity.com/paper/2111.05193/) (2021)
- [Great Power, Great Responsibility: Recommendations for Reducing Energy for Training Language Models](https://mlsanity.com/paper/2205.09646/) (2022)
- [WATCH: Wasserstein Change Point Detection for High-Dimensional Time Series Data](https://mlsanity.com/paper/2201.07125/) (2022)
- [Reconsidering CO2 emissions from Computer Vision](https://mlsanity.com/paper/2104.08702/) (2021)
- [Towards Inadequately Pre-trained Models in Transfer Learning](https://mlsanity.com/paper/2203.04668/) (2022)
- [Aggregate Representation Measure for Predictive Model Reusability](https://mlsanity.com/paper/2405.09600/) (2024)

## Built on by (most cited first, 2 of 2)

- [RESQUE: Quantifying Estimator to Task and Distribution Shift for Sustainable Model Reusability](https://mlsanity.com/paper/2412.15511/) (2024)
- [Evaluation of ML Resource Utilization Requires Model Life Cycle Assessment](https://mlsanity.com/paper/2606.07632/) (2026)

## Abstract (arXiv)

With the rapid increase in the research, development, and application of neural networks in the current era, there is a proportional increase in the energy needed to train and use models. Crucially, this is accompanied by the increase in carbon emissions into the environment. A sustainable and socially beneficial approach to reducing the carbon footprint and rising energy demands associated with the modern age of AI/deep learning is the adaptive and continuous reuse of models with regard to changes in the environment of model deployment or variations/changes in the input data. In this paper, we propose PreIndex, a predictive index to estimate the environmental and compute resources associated with model retraining to distributional shifts in data. PreIndex can be used to estimate environmental costs such as carbon emissions and energy usage when retraining from current data distribution to new data distribution. It also correlates with and can be used to estimate other resource indicators associated with deep learning, such as epochs, gradient norm, and magnitude of model parameter change. PreIndex requires only one forward pass of the data, following which it provides a single concise value to estimate resources associated with retraining to the new distribution shifted data. We show that PreIndex can be reliably used across various datasets, model architectures, different types, and intensities of distribution shifts. Thus, PreIndex enables users to make informed decisions for retraining to different distribution shifts and determine the most cost-effective and sustainable option, allowing for the reuse of a model with a much smaller footprint in the environment. The code for this work is available here: https://github.com/JEKimLab/AIES2024PreIndex