# A Topological Improvement of the Overall Performance of Sparse Evolutionary Training: Motif-Based Structural Optimization of Sparse MLPs Project

arXiv: https://arxiv.org/abs/2506.09204 · HTML: https://mlsanity.com/paper/2506.09204/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 29 references that are in the map.
- First posted on arXiv 10 Jun 2025.

## Builds on (most cited first, 13 of 13)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [Scalable Training of Artificial Neural Networks with Adaptive Sparse Connectivity inspired by Network Science](https://mlsanity.com/paper/1707.04780/) (2017)
- [Parameter Efficient Training of Deep Convolutional Neural Networks by Dynamic Sparse Reparameterization](https://mlsanity.com/paper/1902.05967/) (2019)
- [Deep Rewiring: Training very sparse deep networks](https://mlsanity.com/paper/1711.05136/) (2017)
- [Performance-Efficiency Trade-offs in Unsupervised Pre-training for Speech Recognition](https://mlsanity.com/paper/2109.06870/) (2021)
- [On the Dimensionality of Word Embedding](https://mlsanity.com/paper/1812.04224/) (2018)
- [Testing Deep Neural Networks](https://mlsanity.com/paper/1803.04792/) (2018)
- [Quick and Robust Feature Selection: the Strength of Energy-efficient Sparse Training for Autoencoders](https://mlsanity.com/paper/2012.00560/) (2020)
- [Sparsely-Connected Neural Networks: Towards Efficient VLSI Implementation of Deep Neural Networks](https://mlsanity.com/paper/1611.01427/) (2016)
- [Topology Aware Deep Learning for Wireless Network Optimization](https://mlsanity.com/paper/1912.08336/) (2019)
- [Understanding MLP-Mixer as a Wide and Sparse MLP](https://mlsanity.com/paper/2306.01470/) (2023)
- [Rethinking Diversity in Deep Neural Network Testing](https://mlsanity.com/paper/2305.15698/) (2023)

## Abstract (arXiv)

Deep Neural Networks (DNNs) have been proven to be exceptionally effective and have been applied across diverse domains within deep learning. However, as DNN models increase in complexity, the demand for reduced computational costs and memory overheads has become increasingly urgent. Sparsity has emerged as a leading approach in this area. The robustness of sparse Multi-layer Perceptrons (MLPs) for supervised feature selection, along with the application of Sparse Evolutionary Training (SET), illustrates the feasibility of reducing computational costs without compromising accuracy. Moreover, it is believed that the SET algorithm can still be improved through a structural optimization method called motif-based optimization, with potential efficiency gains exceeding 40% and a performance decline of under 4%. This research investigates whether the structural optimization of Sparse Evolutionary Training applied to Multi-layer Perceptrons (SET-MLP) can enhance performance and to what extent this improvement can be achieved.