# Fine-Pruning: Joint Fine-Tuning and Compression of a Convolutional Network with Bayesian Optimization

arXiv: https://arxiv.org/abs/1707.09102 · HTML: https://mlsanity.com/paper/1707.09102/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jul 2017.

## Built on by (most cited first, 8 of 8)

- [Bayesian Optimization with Clustering and Rollback for CNN Auto Pruning](https://mlsanity.com/paper/2109.10591/) (2021)
- [B-FPGM: Lightweight Face Detection via Bayesian-Optimized Soft FPGM Pruning](https://mlsanity.com/paper/2501.16917/) (2025)
- [Learning Pruned Structure and Weights Simultaneously from Scratch: an Attention based Approach](https://mlsanity.com/paper/2111.02399/) (2021)
- [Design Automation for Fast, Lightweight, and Effective Deep Learning Models: A Survey](https://mlsanity.com/paper/2208.10498/) (2022)
- [Surrogate Lagrangian Relaxation: A Path To Retrain-free Deep Neural Network Pruning](https://mlsanity.com/paper/2304.04120/) (2023)
- [ClearMark: Intuitive and Robust Model Watermarking via Transposed Model Training](https://mlsanity.com/paper/2310.16453/) (2023)
- [Mobile Fitting Room: On-device Virtual Try-on via Diffusion Models](https://mlsanity.com/paper/2402.01877/) (2024)
- [TuneComp: Joint Fine-tuning and Compression for Large Foundation Models](https://mlsanity.com/paper/2505.21835/) (2025)

## Abstract (arXiv)

When approaching a novel visual recognition problem in a specialized image domain, a common strategy is to start with a pre-trained deep neural network and fine-tune it to the specialized domain. If the target domain covers a smaller visual space than the source domain used for pre-training (e.g. ImageNet), the fine-tuned network is likely to be over-parameterized. However, applying network pruning as a post-processing step to reduce the memory requirements has drawbacks: fine-tuning and pruning are performed independently; pruning parameters are set once and cannot adapt over time; and the highly parameterized nature of state-of-the-art pruning methods make it prohibitive to manually search the pruning parameter space for deep networks, leading to coarse approximations. We propose a principled method for jointly fine-tuning and compressing a pre-trained convolutional network that overcomes these limitations. Experiments on two specialized image domains (remote sensing images and describable textures) demonstrate the validity of the proposed approach.