# Lift What You Can: Green Online Learning with Heterogeneous Ensembles

arXiv: https://arxiv.org/abs/2509.18962 · HTML: https://mlsanity.com/paper/2509.18962/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 40 references that are in the map.
- First posted on arXiv 23 Sep 2025.

## Builds on (most cited first, 10 of 10)

- [Green AI](https://mlsanity.com/paper/1907.10597/) (2019)
- [Learning under Concept Drift: A Review](https://mlsanity.com/paper/2004.05785/) (2020)
- [Efficient Online Reinforcement Learning with Offline Data](https://mlsanity.com/paper/2302.02948/) (2023)
- [Power Hungry Processing: Watts Driving the Cost of AI Deployment?](https://mlsanity.com/paper/2311.16863/) (2023)
- [Extremely Fast Decision Tree](https://mlsanity.com/paper/1802.08780/) (2018)
- [Hype, Sustainability, and the Price of the Bigger-is-Better Paradigm in AI](https://mlsanity.com/paper/2409.14160/) (2024)
- [Online Gradient Boosting](https://mlsanity.com/paper/1506.04820/) (2015)
- [AutoXPCR: Automated Multi-Objective Model Selection for Time Series Forecasting](https://mlsanity.com/paper/2312.13038/) (2023)
- [Shrub Ensembles for Online Classification](https://mlsanity.com/paper/2112.03723/) (2021)
- [Soft Hoeffding Tree: A Transparent and Differentiable Model on Data Streams](https://mlsanity.com/paper/2411.04812/) (2024)

## Built on by (most cited first, 1 of 1)

- [Pruning Extensions and Efficiency Trade-Offs for Sustainable Time Series Classification](https://mlsanity.com/paper/2604.07953/) (2026)

## Abstract (arXiv)

Ensemble methods for stream mining necessitate managing multiple models and updating them as data distributions evolve. Considering the calls for more sustainability, established methods are however not sufficiently considerate of ensemble members' computational expenses and instead overly focus on predictive capabilities. To address these challenges and enable green online learning, we propose heterogeneous online ensembles (HEROS). For every training step, HEROS chooses a subset of models from a pool of models initialized with diverse hyperparameter choices under resource constraints to train. We introduce a Markov decision process to theoretically capture the trade-offs between predictive performance and sustainability constraints. Based on this framework, we present different policies for choosing which models to train on incoming data. Most notably, we propose the novel zeta-policy, which focuses on training near-optimal models at reduced costs. Using a stochastic model, we theoretically prove that our zeta-policy achieves near optimal performance while using fewer resources compared to the best performing policy. In our experiments across 11 benchmark datasets, we find empiric evidence that our zeta-policy is a strong contribution to the state-of-the-art, demonstrating highly accurate performance, in some cases even outperforming competitors, and simultaneously being much more resource-friendly.