# Identifying architectural design decisions for achieving green ML serving

arXiv: https://arxiv.org/abs/2402.07585 · HTML: https://mlsanity.com/paper/2402.07585/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 27 references that are in the map.
- First posted on arXiv 12 Feb 2024.

## Builds on (most cited first, 9 of 9)

- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [Green AI](https://mlsanity.com/paper/1907.10597/) (2019)
- [Sustainable AI: Environmental Implications, Challenges and Opportunities](https://mlsanity.com/paper/2111.00364/) (2021)
- [MLPerf Inference Benchmark](https://mlsanity.com/paper/1911.02549/) (2019)
- [A Systematic Review of Green AI](https://mlsanity.com/paper/2301.11047/) (2023)
- [Compute and Energy Consumption Trends in Deep Learning Inference](https://mlsanity.com/paper/2109.05472/) (2021)
- [pCAMP: Performance Comparison of Machine Learning Packages on the Edges](https://mlsanity.com/paper/1906.01878/) (2019)
- [Batching for Green AI -- An Exploratory Study on Inference](https://mlsanity.com/paper/2307.11434/) (2023)
- [A Review of Software Quality Models for the Evaluation of Software Products](https://mlsanity.com/paper/1412.2977/) (2014)

## Built on by (most cited first, 2 of 2)

- [Insights into resource utilization of code small language models serving with runtime engines and execution providers](https://mlsanity.com/paper/2412.15441/) (2024)
- [The Energy Impact of Domain Model Design in Classical Planning](https://mlsanity.com/paper/2601.21967/) (2026)

## Abstract (arXiv)

The growing use of large machine learning models highlights concerns about their increasing computational demands. While the energy consumption of their training phase has received attention, fewer works have considered the inference phase. For ML inference, the binding of ML models to the ML system for user access, known as ML serving, is a critical yet understudied step for achieving efficiency in ML applications. We examine the literature in ML architectural design decisions and Green AI, with a special focus on ML serving. The aim is to analyze ML serving architectural design decisions for the purpose of understanding and identifying them with respect to quality characteristics from the point of view of researchers and practitioners in the context of ML serving literature. Our results (i) identify ML serving architectural design decisions along with their corresponding components and associated technological stack, and (ii) provide an overview of the quality characteristics studied in the literature, including energy efficiency. This preliminary study is the first step in our goal to achieve green ML serving. Our analysis may aid ML researchers and practitioners in making green-aware architecture design decisions when serving their models.