# The More the Merrier? Navigating Accuracy vs. Energy Efficiency Design Trade-Offs in Ensemble Learning Systems

arXiv: https://arxiv.org/abs/2407.02914 · HTML: https://mlsanity.com/paper/2407.02914/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 63 references that are in the map.
- First posted on arXiv 3 Jul 2024.

## Builds on (most cited first, 17 of 17)

- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [Green AI](https://mlsanity.com/paper/1907.10597/) (2019)
- [A Systematic Review of Green AI](https://mlsanity.com/paper/2301.11047/) (2023)
- [A Survey on Ensemble Learning under the Era of Deep Learning](https://mlsanity.com/paper/2101.08387/) (2021)
- [Data-Centric Green AI: An Exploratory Empirical Study](https://mlsanity.com/paper/2204.02766/) (2022)
- [A Synthesis of Green Architectural Tactics for ML-Enabled Systems](https://mlsanity.com/paper/2312.09610/) (2023)
- [Energy Efficiency of Training Neural Network Architectures: An Empirical Study](https://mlsanity.com/paper/2302.00967/) (2023)
- [Machine Learning Towards Intelligent Systems: Applications, Challenges, and Opportunities](https://mlsanity.com/paper/2101.03655/) (2021)
- [Towards Inference Efficient Deep Ensemble Learning](https://mlsanity.com/paper/2301.12378/) (2023)
- [Differentiable Model Selection for Ensemble Learning](https://mlsanity.com/paper/2211.00251/) (2022)
- [Feature Interactions on Steroids: On the Composition of ML Models](https://mlsanity.com/paper/2105.06449/) (2021)
- [A Compositional Approach to Creating Architecture Frameworks with an Application to Distributed AI Systems](https://mlsanity.com/paper/2212.13570/) (2022)
- [Estimating Deep Learning energy consumption based on model architecture and training environment](https://mlsanity.com/paper/2307.05520/) (2023)
- [Training Green AI Models Using Elite Samples](https://mlsanity.com/paper/2402.12010/) (2024)
- [EMaaS: Energy Measurements as a Service for Mobile Applications](https://mlsanity.com/paper/1902.02605/) (2019)
- [How to Sustainably Monitor ML-Enabled Systems? Accuracy and Energy Efficiency Tradeoffs in Concept Drift Detection](https://mlsanity.com/paper/2404.19452/) (2024)
- [A post-selection algorithm for improving dynamic ensemble selection methods](https://mlsanity.com/paper/2309.14307/) (2023)

## Built on by (most cited first, 1 of 1)

- [Choosing to Be Green: Advancing Green AI via Dynamic Model Selection](https://mlsanity.com/paper/2509.19996/) (2025)

## Abstract (arXiv)

Background: Machine learning (ML) model composition is a popular technique to mitigate shortcomings of a single ML model and to design more effective ML-enabled systems. While ensemble learning, i.e., forwarding the same request to several models and fusing their predictions, has been studied extensively for accuracy, we have insufficient knowledge about how to design energy-efficient ensembles. Objective: We therefore analyzed three types of design decisions for ensemble learning regarding a potential trade-off between accuracy and energy consumption: a) ensemble size, i.e., the number of models in the ensemble, b) fusion methods (majority voting vs. a meta-model), and c) partitioning methods (whole-dataset vs. subset-based training). Methods: By combining four popular ML algorithms for classification in different ensembles, we conducted a full factorial experiment with 11 ensembles x 4 datasets x 2 fusion methods x 2 partitioning methods (176 combinations). For each combination, we measured accuracy (F1-score) and energy consumption in J (for both training and inference). Results: While a larger ensemble size significantly increased energy consumption (size 2 ensembles consumed 37.49% less energy than size 3 ensembles, which in turn consumed 26.96% less energy than the size 4 ensembles), it did not significantly increase accuracy. Furthermore, majority voting outperformed meta-model fusion both in terms of accuracy (Cohen's d of 0.38) and energy consumption (Cohen's d of 0.92). Lastly, subset-based training led to significantly lower energy consumption (Cohen's d of 0.91), while training on the whole dataset did not increase accuracy significantly. Conclusions: From a Green AI perspective, we recommend designing ensembles of small size (2 or maximum 3 models), using subset-based training, majority voting, and energy-efficient ML algorithms like decision trees, Naive Bayes, or KNN.