# Projection-Free CNN Pruning via Frank-Wolfe with Momentum: Sparser Models with Less Pretraining

arXiv: https://arxiv.org/abs/2512.01147 · HTML: https://mlsanity.com/paper/2512.01147/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 15 references that are in the map.
- First posted on arXiv 30 Nov 2025.

## Builds on (most cited first, 9 of 9)

- [Rethinking the Value of Network Pruning](https://mlsanity.com/paper/1810.05270/) (2018)
- [What is the State of Neural Network Pruning?](https://mlsanity.com/paper/2003.03033/) (2020)
- [Good Subnetworks Provably Exist: Pruning via Greedy Forward Selection](https://mlsanity.com/paper/2003.01794/) (2020)
- [Deep Neural Network Training with Frank-Wolfe](https://mlsanity.com/paper/2010.07243/) (2020)
- [Layer-compensated Pruning for Resource-constrained Convolutional Neural Networks](https://mlsanity.com/paper/1810.00518/) (2018)
- [The Frank-Wolfe algorithm: a short introduction](https://mlsanity.com/paper/2311.05313/) (2023)
- [How much pre-training is enough to discover a good subnetwork?](https://mlsanity.com/paper/2108.00259/) (2021)
- [Frank-Wolfe Style Algorithms for Large Scale Optimization](https://mlsanity.com/paper/1808.05274/) (2018)
- [Frank-Wolfe Methods with an Unbounded Feasible Region and Applications to Structured Learning](https://mlsanity.com/paper/2012.15361/) (2020)

## Abstract (arXiv)

We investigate algorithmic variants of the Frank-Wolfe (FW) optimization method for pruning convolutional neural networks. This is motivated by the "Lottery Ticket Hypothesis", which suggests the existence of smaller sub-networks within larger pre-trained networks that perform comparatively well (if not better). Whilst most literature in this area focuses on Deep Neural Networks more generally, we specifically consider Convolutional Neural Networks for image classification tasks. Building on the hypothesis, we compare simple magnitude-based pruning, a Frank-Wolfe style pruning scheme, and an FW method with momentum on a CNN trained on MNIST. Our experiments track test accuracy, loss, sparsity, and inference time as we vary the dense pre-training budget from 1 to 10 epochs. We find that FW with momentum yields pruned networks that are both sparser and more accurate than the original dense model and the simple pruning baselines, while incurring minimal inference-time overhead in our implementation. Moreover, FW with momentum reaches these accuracies after only a few epochs of pre-training, indicating that full pre-training of the dense model is not required in this setting.