# Distilling Critical Paths in Convolutional Neural Networks

arXiv: https://arxiv.org/abs/1811.02643 · HTML: https://mlsanity.com/paper/1811.02643/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Oct 2018.

## Built on by (most cited first, 7 of 7)

- [Fed2: Feature-Aligned Federated Learning](https://mlsanity.com/paper/2111.14248/) (2021)
- [Neural Response Interpretation through the Lens of Critical Pathways](https://mlsanity.com/paper/2103.16886/) (2021)
- [FedSkel: Efficient Federated Learning on Heterogeneous Systems with Skeleton Gradients Update](https://mlsanity.com/paper/2108.09081/) (2021)
- [On Model Explanations with Transferable Neural Pathways](https://mlsanity.com/paper/2309.09887/) (2023)
- [Canoe : A System for Collaborative Learning for Neural Nets](https://mlsanity.com/paper/2108.12124/) (2021)
- [Class-Wise Federated Averaging for Efficient Personalization](https://mlsanity.com/paper/2406.07800/) (2024)
- [Anomaly Detection Based on Critical Paths for Deep Neural Networks](https://mlsanity.com/paper/2505.14967/) (2025)

## Abstract (arXiv)

Neural network compression and acceleration are widely demanded currently due to the resource constraints on most deployment targets. In this paper, through analyzing the filter activation, gradients, and visualizing the filters' functionality in convolutional neural networks, we show that the filters in higher layers learn extremely task-specific features, which are exclusive for only a small subset of the overall tasks, or even a single class. Based on such findings, we reveal the critical paths of information flow for different classes. And by their intrinsic property of exclusiveness, we propose a critical path distillation method, which can effectively customize the convolutional neural networks to small ones with much smaller model size and less computation.