# Frugal Machine Learning for Energy-efficient, and Resource-aware Artificial Intelligence

arXiv: https://arxiv.org/abs/2506.01869 · HTML: https://mlsanity.com/paper/2506.01869/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 81 references that are in the map.
- First posted on arXiv 2 Jun 2025.

## Builds on (most cited first, 30 of 37)

- [The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://mlsanity.com/paper/1803.03635/) (2018)
- [Pruning Filters for Efficient ConvNets](https://mlsanity.com/paper/1608.08710/) (2016)
- [Once-for-All: Train One Network and Specialize it for Efficient Deployment](https://mlsanity.com/paper/1908.09791/) (2019)
- [Mixture-of-Experts with Expert Choice Routing](https://mlsanity.com/paper/2202.09368/) (2022)
- [Slimmable Neural Networks](https://mlsanity.com/paper/1812.08928/) (2018)
- [Model compression via distillation and quantization](https://mlsanity.com/paper/1802.05668/) (2018)
- [Dynamic Network Surgery for Efficient DNNs](https://mlsanity.com/paper/1608.04493/) (2016)
- [A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendations](https://mlsanity.com/paper/2308.06767/) (2023)
- [Dataset Distillation: A Comprehensive Review](https://mlsanity.com/paper/2301.07014/) (2023)
- [Structured Pruning for Deep Convolutional Neural Networks: A survey](https://mlsanity.com/paper/2303.00566/) (2023)
- [Dynamic Convolutions: Exploiting Spatial Sparsity for Faster Inference](https://mlsanity.com/paper/1912.03203/) (2019)
- [A Survey of Neuromorphic Computing and Neural Networks in Hardware](https://mlsanity.com/paper/1705.06963/) (2017)
- [Rethinking Kullback-Leibler Divergence in Knowledge Distillation for Large Language Models](https://mlsanity.com/paper/2404.02657/) (2024)
- [Lost in Pruning: The Effects of Pruning Neural Networks beyond Test Accuracy](https://mlsanity.com/paper/2103.03014/) (2021)
- [Dynamic Deep Neural Networks: Optimizing Accuracy-Efficiency Trade-offs by Selective Execution](https://mlsanity.com/paper/1701.00299/) (2017)
- [Learning Low-rank Deep Neural Networks via Singular Vector Orthogonality Regularization and Singular Value Sparsification](https://mlsanity.com/paper/2004.09031/) (2020)
- [Inverse Optimization: Theory and Applications](https://mlsanity.com/paper/2109.03920/) (2021)
- [Relief-Based Feature Selection: Introduction and Review](https://mlsanity.com/paper/1711.08421/) (2017)
- [Fire Together Wire Together: A Dynamic Pruning Approach with Self-Supervised Mask Prediction](https://mlsanity.com/paper/2110.08232/) (2021)
- [Collaborative Multi-Teacher Knowledge Distillation for Learning Low Bit-width Deep Neural Networks](https://mlsanity.com/paper/2210.16103/) (2022)
- [Categories of Response-Based, Feature-Based, and Relation-Based Knowledge Distillation](https://mlsanity.com/paper/2306.10687/) (2023)
- [Improving the Efficiency of Transformers for Resource-Constrained Devices](https://mlsanity.com/paper/2106.16006/) (2021)
- [SlimConv: Reducing Channel Redundancy in Convolutional Neural Networks by Weights Flipping](https://mlsanity.com/paper/2003.07469/) (2020)
- [Mapping Spiking Neural Networks to Neuromorphic Hardware](https://mlsanity.com/paper/1909.01843/) (2019)
- [Development of Skip Connection in Deep Neural Networks for Computer Vision and Medical Image Analysis: A Survey](https://mlsanity.com/paper/2405.01725/) (2024)
- [A novel embedded min-max approach for feature selection in nonlinear support vector machine classification](https://mlsanity.com/paper/2004.09863/) (2020)
- [TransformerG2G: Adaptive time-stepping for learning temporal graph embeddings using transformers](https://mlsanity.com/paper/2307.02588/) (2023)
- [IGRF-RFE: A Hybrid Feature Selection Method for MLP-based Network Intrusion Detection on UNSW-NB15 Dataset](https://mlsanity.com/paper/2203.16365/) (2022)
- [Efficient Learning of Accurate Surrogates for Simulations of Complex Systems](https://mlsanity.com/paper/2207.12855/) (2022)
- [A survey on FPGA-based accelerator for ML models](https://mlsanity.com/paper/2412.15666/) (2024)

## Built on by (most cited first, 1 of 1)

- [Position: Stop Preaching and Start Practising Data Frugality for Responsible Development of AI](https://mlsanity.com/paper/2602.19789/) (2026)

## Abstract (arXiv)

Frugal Machine Learning (FML) refers to the practice of designing Machine Learning (ML) models that are efficient, cost-effective, and mindful of resource constraints. This field aims to achieve acceptable performance while minimizing the use of computational resources, time, energy, and data for both training and inference. FML strategies can be broadly categorized into input frugality, learning process frugality, and model frugality, each focusing on reducing resource consumption at different stages of the ML pipeline. This chapter explores recent advancements, applications, and open challenges in FML, emphasizing its importance for smart environments that incorporate edge computing and IoT devices, which often face strict limitations in bandwidth, energy, or latency. Technological enablers such as model compression, energy-efficient hardware, and data-efficient learning techniques are discussed, along with adaptive methods including parameter regularization, knowledge distillation, and dynamic architecture design that enable incremental model updates without full retraining. Furthermore, it provides a comprehensive taxonomy of frugal methods, discusses case studies across diverse domains, and identifies future research directions to drive innovation in this evolving field.