# Lookahead: A Far-Sighted Alternative of Magnitude-based Pruning

arXiv: https://arxiv.org/abs/2002.04809 · HTML: https://mlsanity.com/paper/2002.04809/

## Key facts

- Cited by 61 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Feb 2020.

## Built on by (most cited first, 30 of 61)

- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [DepGraph: Towards Any Structural Pruning](https://mlsanity.com/paper/2301.12900/) (2023)
- [Structural Pruning for Diffusion Models](https://mlsanity.com/paper/2305.10924/) (2023)
- [Learned Token Pruning for Transformers](https://mlsanity.com/paper/2107.00910/) (2021)
- [Addressing Loss of Plasticity and Catastrophic Forgetting in Continual Learning](https://mlsanity.com/paper/2404.00781/) (2024)
- [Hessian-Aware Pruning and Optimal Neural Implant](https://mlsanity.com/paper/2101.08940/) (2021)
- [DiP-GO: A Diffusion Pruner via Few-step Gradient Optimization](https://mlsanity.com/paper/2410.16942/) (2024)
- [Applications and Techniques for Fast Machine Learning in Science](https://mlsanity.com/paper/2110.13041/) (2021)
- [DKM: Differentiable K-Means Clustering Layer for Neural Network Compression](https://mlsanity.com/paper/2108.12659/) (2021)
- [GDP: Stabilized Neural Network Pruning via Gates with Differentiable Polarization](https://mlsanity.com/paper/2109.02220/) (2021)
- [An Information-Theoretic Justification for Model Pruning](https://mlsanity.com/paper/2102.08329/) (2021)
- [SlimSAM: 0.1% Data Makes Segment Anything Slim](https://mlsanity.com/paper/2312.05284/) (2023)
- [RED : Looking for Redundancies for Data-Free Structured Compression of Deep Neural Networks](https://mlsanity.com/paper/2105.14797/) (2021)
- [RED++ : Data-Free Pruning of Deep Neural Networks via Input Splitting and Output Merging](https://mlsanity.com/paper/2110.01397/) (2021)
- [Agentic World Modeling: Foundations, Capabilities, Laws, and Beyond](https://mlsanity.com/paper/2604.22748/) (2026)
- [Is Complexity Required for Neural Network Pruning? A Case Study on Global Magnitude Pruning](https://mlsanity.com/paper/2209.14624/) (2022)
- [AMD: Automatic Multi-step Distillation of Large-scale Vision Models](https://mlsanity.com/paper/2407.04208/) (2024)
- [MKQ-BERT: Quantized BERT with 4-bits Weights and Activations](https://mlsanity.com/paper/2203.13483/) (2022)
- [F3-Pruning: A Training-Free and Generalized Pruning Strategy towards Faster and Finer Text-to-Video Synthesis](https://mlsanity.com/paper/2312.03459/) (2023)
- [SInGE: Sparsity via Integrated Gradients Estimation of Neuron Relevance](https://mlsanity.com/paper/2207.04089/) (2022)
- [On-device edge learning for IoT data streams: a survey](https://mlsanity.com/paper/2502.17788/) (2025)
- [LEAP: Learnable Pruning for Transformer-based Models](https://mlsanity.com/paper/2105.14636/) (2021)
- [Perturbation of Deep Autoencoder Weights for Model Compression and Classification of Tabular Data](https://mlsanity.com/paper/2205.08358/) (2022)
- [Data-independent Module-aware Pruning for Hierarchical Vision Transformers](https://mlsanity.com/paper/2404.13648/) (2024)
- [Utility-based Perturbed Gradient Descent: An Optimizer for Continual Learning](https://mlsanity.com/paper/2302.03281/) (2023)
- [FastLightGen: Fast and Light Video Generation with Fewer Steps and Parameters](https://mlsanity.com/paper/2603.01685/) (2026)
- [AdaptCL: Efficient Collaborative Learning with Dynamic and Adaptive Pruning](https://mlsanity.com/paper/2106.14126/) (2021)
- [eDKM: An Efficient and Accurate Train-time Weight Clustering for Large Language Models](https://mlsanity.com/paper/2309.00964/) (2023)
- [PriPrune: Quantifying and Preserving Privacy in Pruned Federated Learning](https://mlsanity.com/paper/2310.19958/) (2023)
- [ALTER: All-in-One Layer Pruning and Temporal Expert Routing for Efficient Diffusion Generation](https://mlsanity.com/paper/2505.21817/) (2025)

## Abstract (arXiv)

Magnitude-based pruning is one of the simplest methods for pruning neural networks. Despite its simplicity, magnitude-based pruning and its variants demonstrated remarkable performances for pruning modern architectures. Based on the observation that magnitude-based pruning indeed minimizes the Frobenius distortion of a linear operator corresponding to a single layer, we develop a simple pruning method, coined lookahead pruning, by extending the single layer optimization to a multi-layer optimization. Our experimental results demonstrate that the proposed method consistently outperforms magnitude-based pruning on various networks, including VGG and ResNet, particularly in the high-sparsity regime. See https://github.com/alinlab/lookahead_pruning for codes.