# Aggregate Representation Measure for Predictive Model Reusability

arXiv: https://arxiv.org/abs/2405.09600 · HTML: https://mlsanity.com/paper/2405.09600/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 25 references that are in the map.
- First posted on arXiv 15 May 2024.

## Builds on (most cited first, 19 of 19)

- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty](https://mlsanity.com/paper/1912.02781/) (2019)
- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [Quantifying the Carbon Emissions of Machine Learning](https://mlsanity.com/paper/1910.09700/) (2019)
- [A Fourier Perspective on Model Robustness in Computer Vision](https://mlsanity.com/paper/1906.08988/) (2019)
- [Puzzle Mix: Exploiting Saliency and Local Statistics for Optimal Mixup](https://mlsanity.com/paper/2009.06962/) (2020)
- [Generalisation in humans and deep neural networks](https://mlsanity.com/paper/1808.08750/) (2018)
- [Carbontracker: Tracking and Predicting the Carbon Footprint of Training Deep Learning Models](https://mlsanity.com/paper/2007.03051/) (2020)
- [Designing Energy-Efficient Convolutional Neural Networks using Energy-Aware Pruning](https://mlsanity.com/paper/1611.05128/) (2016)
- [Adversarial Examples Are a Natural Consequence of Test Error in Noise](https://mlsanity.com/paper/1901.10513/) (2019)
- [TTN: A Domain-Shift Aware Batch Normalization in Test-Time Adaptation](https://mlsanity.com/paper/2302.05155/) (2023)
- [On Robustness and Transferability of Convolutional Neural Networks](https://mlsanity.com/paper/2007.08558/) (2020)
- [Test-Time Adaptation via Conjugate Pseudo-labels](https://mlsanity.com/paper/2207.09640/) (2022)
- [Estimating Example Difficulty Using Variance of Gradients](https://mlsanity.com/paper/2008.11600/) (2020)
- [The Evolution of Out-of-Distribution Robustness Throughout Fine-Tuning](https://mlsanity.com/paper/2106.15831/) (2021)
- [A Survey on Green Deep Learning](https://mlsanity.com/paper/2111.05193/) (2021)
- [A Systematic Review of Robustness in Deep Learning for Computer Vision: Mind the gap?](https://mlsanity.com/paper/2112.00639/) (2021)

## Built on by (most cited first, 2 of 2)

- [Estimating Environmental Cost Throughout Model's Adaptive Life Cycle](https://mlsanity.com/paper/2408.01446/) (2024)
- [RESQUE: Quantifying Estimator to Task and Distribution Shift for Sustainable Model Reusability](https://mlsanity.com/paper/2412.15511/) (2024)

## Abstract (arXiv)

In this paper, we propose a predictive quantifier to estimate the retraining cost of a trained model in distribution shifts. The proposed Aggregated Representation Measure (ARM) quantifies the change in the model's representation from the old to new data distribution. It provides, before actually retraining the model, a single concise index of resources - epochs, energy, and carbon emissions - required for the retraining. This enables reuse of a model with a much lower cost than training a new model from scratch. The experimental results indicate that ARM reasonably predicts retraining costs for varying noise intensities and enables comparisons among multiple model architectures to determine the most cost-effective and sustainable option.