# Automated Evolutionary Optimization for Resource-Efficient Neural Network Training

arXiv: https://arxiv.org/abs/2510.09566 · HTML: https://mlsanity.com/paper/2510.09566/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 40 references that are in the map.
- First posted on arXiv 10 Oct 2025.

## Builds on (most cited first, 25 of 25)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [Learning Efficient Convolutional Networks through Network Slimming](https://mlsanity.com/paper/1708.06519/) (2017)
- [Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey](https://mlsanity.com/paper/2403.14608/) (2024)
- [Importance Estimation for Neural Network Pruning](https://mlsanity.com/paper/1906.10771/) (2019)
- [Pruning and Quantization for Deep Neural Network Acceleration: A Survey](https://mlsanity.com/paper/2101.09671/) (2021)
- [UniPELT: A Unified Framework for Parameter-Efficient Language Model Tuning](https://mlsanity.com/paper/2110.07577/) (2021)
- [Layer-adaptive sparsity for the Magnitude-based Pruning](https://mlsanity.com/paper/2010.07611/) (2020)
- [Tracking Meets LoRA: Faster Training, Larger Model, Stronger Performance](https://mlsanity.com/paper/2403.05231/) (2024)
- [AutoML to Date and Beyond: Challenges and Opportunities](https://mlsanity.com/paper/2010.10777/) (2020)
- [CoLES: Contrastive Learning for Event Sequences with Self-Supervision](https://mlsanity.com/paper/2002.08232/) (2020)
- [LightAutoML: AutoML Solution for a Large Financial Services Ecosystem](https://mlsanity.com/paper/2109.01528/) (2021)
- [Latency and Throughput Characterization of Convolutional Neural Networks for Mobile Computer Vision](https://mlsanity.com/paper/1803.09492/) (2018)
- [Multi-Objective Evolutionary Design of Composite Data-Driven Models](https://mlsanity.com/paper/2103.01301/) (2021)
- [FFSplit: Split Feed-Forward Network For Optimizing Accuracy-Efficiency Trade-off in Language Model Inference](https://mlsanity.com/paper/2401.04044/) (2024)
- [From PEFT to DEFT: Parameter Efficient Finetuning for Reducing Activation Density in Transformers](https://mlsanity.com/paper/2402.01911/) (2024)
- [Model Uncertainty in Evolutionary Optimization and Bayesian Optimization: A Comparative Analysis](https://mlsanity.com/paper/2403.14413/) (2024)
- [Preserving Pre-trained Representation Space: On Effectiveness of Prefix-tuning for Large Multi-modal Models](https://mlsanity.com/paper/2411.00029/) (2024)
- [Preprint: Norm Loss: An efficient yet effective regularization method for deep neural networks](https://mlsanity.com/paper/2103.06583/) (2021)
- [Regularization-based Pruning of Irrelevant Weights in Deep Neural Architectures](https://mlsanity.com/paper/2204.04977/) (2022)
- [Improvement of Computational Performance of Evolutionary AutoML in a Heterogeneous Environment](https://mlsanity.com/paper/2301.05102/) (2023)
- [Lai Loss: A Novel Loss for Gradient Control](https://mlsanity.com/paper/2405.07884/) (2024)
- [Enhancing Parameter-Efficient Fine-Tuning of Vision Transformers through Frequency-Based Adaptation](https://mlsanity.com/paper/2411.19297/) (2024)

## Abstract (arXiv)

There are many critical challenges in optimizing neural network models, including distributed computing, compression techniques, and efficient training, regardless of their application to specific tasks. Solving such problems is crucial because the need for scalable and resource-efficient models is increasing. To address these challenges, we have developed a new automated machine learning (AutoML) framework, Parameter Efficient Training with Robust Automation (PETRA). It applies evolutionary optimization to model architecture and training strategy. PETRA includes pruning, quantization, and loss regularization. Experimental studies on real-world data with financial event sequences, as well as image and time-series -- benchmarks, demonstrate PETRA's ability to improve neural model performance and scalability -- namely, a significant decrease in model size (up to 75%) and latency (up to 33%), and an increase in throughput (by 13%) without noticeable degradation in the target metric.