# FLOPs as a Direct Optimization Objective for Learning Sparse Neural Networks

arXiv: https://arxiv.org/abs/1811.03060 · HTML: https://mlsanity.com/paper/1811.03060/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Nov 2018.

## Built on by (most cited first, 7 of 7)

- [Kolmogorov-Arnold Fourier Networks](https://mlsanity.com/paper/2502.06018/) (2025)
- [FALCON: FLOP-Aware Combinatorial Optimization for Neural Network Pruning](https://mlsanity.com/paper/2403.07094/) (2024)
- [Shapley Pruning for Neural Network Compression](https://mlsanity.com/paper/2407.15875/) (2024)
- [Robust and Resource-efficient Machine Learning Aided Viewport Prediction in Virtual Reality](https://mlsanity.com/paper/2212.09945/) (2022)
- [Practical Conformer: Optimizing size, speed and flops of Conformer for on-Device and cloud ASR](https://mlsanity.com/paper/2304.00171/) (2023)
- [REFT: Resource-Efficient Federated Training Framework for Heterogeneous and Resource-Constrained Environments](https://mlsanity.com/paper/2308.13662/) (2023)
- [H-FLTN: A Privacy-Preserving Hierarchical Framework for Electric Vehicle Spatio-Temporal Charge Prediction](https://mlsanity.com/paper/2502.18697/) (2025)

## Abstract (arXiv)

There exists a plethora of techniques for inducing structured sparsity in parametric models during the optimization process, with the final goal of resource-efficient inference. However, few methods target a specific number of floating-point operations (FLOPs) as part of the optimization objective, despite many reporting FLOPs as part of the results. Furthermore, a one-size-fits-all approach ignores realistic system constraints, which differ significantly between, say, a GPU and a mobile phone -- FLOPs on the former incur less latency than on the latter; thus, it is important for practitioners to be able to specify a target number of FLOPs during model compression. In this work, we extend a state-of-the-art technique to directly incorporate FLOPs as part of the optimization objective and show that, given a desired FLOPs requirement, different neural networks can be successfully trained for image classification.