# Green Runner: A tool for efficient deep learning component selection

arXiv: https://arxiv.org/abs/2401.15810 · HTML: https://mlsanity.com/paper/2401.15810/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 27 references that are in the map.
- First posted on arXiv 29 Jan 2024.

## Builds on (most cited first, 18 of 18)

- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [Do Better ImageNet Models Transfer Better?](https://mlsanity.com/paper/1805.08974/) (2018)
- [Introduction to Multi-Armed Bandits](https://mlsanity.com/paper/1904.07272/) (2019)
- [Sustainable AI: Environmental Implications, Challenges and Opportunities](https://mlsanity.com/paper/2111.00364/) (2021)
- [LogME: Practical Assessment of Pre-trained Models for Transfer Learning](https://mlsanity.com/paper/2102.11005/) (2021)
- [Measuring the Carbon Intensity of AI in Cloud Instances](https://mlsanity.com/paper/2206.05229/) (2022)
- [Representation Similarity Analysis for Efficient Task taxonomy & Transfer Learning](https://mlsanity.com/paper/1904.11740/) (2019)
- [Towards Learning Universal Hyperparameter Optimizers with Transformers](https://mlsanity.com/paper/2205.13320/) (2022)
- [Transferability Estimation using Bhattacharyya Class Separability](https://mlsanity.com/paper/2111.12780/) (2021)
- [IGLUE: A Benchmark for Transfer Learning across Modalities, Tasks, and Languages](https://mlsanity.com/paper/2201.11732/) (2022)
- [Large Language Models for Compiler Optimization](https://mlsanity.com/paper/2309.07062/) (2023)
- [Frustratingly Easy Transferability Estimation](https://mlsanity.com/paper/2106.09362/) (2021)
- [Scalable Diverse Model Selection for Accessible Transfer Learning](https://mlsanity.com/paper/2111.06977/) (2021)
- [Unraveling the Hidden Environmental Impacts of AI Solutions for Environment](https://mlsanity.com/paper/2110.11822/) (2021)
- [Model Spider: Learning to Rank Pre-Trained Models Efficiently](https://mlsanity.com/paper/2306.03900/) (2023)
- [PACTran: PAC-Bayesian Metrics for Estimating the Transferability of Pretrained Models to Classification Tasks](https://mlsanity.com/paper/2203.05126/) (2022)
- [Enabling Pedestrian Safety using Computer Vision Techniques: A Case Study of the 2018 Uber Inc. Self-driving Car Crash](https://mlsanity.com/paper/1805.11815/) (2018)
- [Threshy: Supporting Safe Usage of Intelligent Web Services](https://mlsanity.com/paper/2008.08252/) (2020)

## Abstract (arXiv)

For software that relies on machine-learned functionality, model selection is key to finding the right model for the task with desired performance characteristics. Evaluating a model requires developers to i) select from many models (e.g. the Hugging face model repository), ii) select evaluation metrics and training strategy, and iii) tailor trade-offs based on the problem domain. However, current evaluation approaches are either ad-hoc resulting in sub-optimal model selection or brute force leading to wasted compute. In this work, we present toolname, a novel tool to automatically select and evaluate models based on the application scenario provided in natural language. We leverage the reasoning capabilities of large language models to propose a training strategy and extract desired trade-offs from a problem description. toolname~features a resource-efficient experimentation engine that integrates constraints and trade-offs based on the problem into the model selection process. Our preliminary evaluation demonstrates that toolname is both efficient and accurate compared to ad-hoc evaluations and brute force. This work presents an important step toward energy-efficient tools to help reduce the environmental impact caused by the growing demand for software with machine-learned functionality.