mlsanityData to 14 Oct 2026

Lookahead: A Far-Sighted Alternative of Magnitude-based Pruning

arXiv 2002.04809 ↗First posted 12 Feb 2020cs.LG stat.MLcited by 61

Not on the map. Why?

Abstract (arXiv)

Magnitude-based pruning is one of the simplest methods for pruning neural networks. Despite its simplicity, magnitude-based pruning and its variants demonstrated remarkable performances for pruning modern architectures. Based on the observation that magnitude-based pruning indeed minimizes the Frobenius distortion of a linear operator corresponding to a single layer, we develop a simple pruning method, coined lookahead pruning, by extending the single layer optimization to a multi-layer optimization. Our experimental results demonstrate that the proposed method consistently outperforms magnitude-based pruning on various networks, including VGG and ResNet, particularly in the high-sparsity regime. See https://github.com/alinlab/lookahead_pruning for codes.

Key facts. Cited by 61 papers in the map (data through 14 Oct 2026). First posted on arXiv 12 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 15; 2022: 9; 2023: 13; 2024: 9; 2025: 8; 2026: 7.

61 papers.

Sort:

By area:

  1. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  2. 2023DepGraph: Towards Any Structural Pruningcited by 203
  3. 2023Structural Pruning for Diffusion Modelscited by 195
  4. 2021Learned Token Pruning for Transformerscited by 139
  5. 2024Addressing Loss of Plasticity and Catastrophic Forgetting in Continual Learningcited by 50
  6. 2021Hessian-Aware Pruning and Optimal Neural Implantcited by 45
  7. 2024DiP-GO: A Diffusion Pruner via Few-step Gradient Optimizationcited by 43
  8. 2021Applications and Techniques for Fast Machine Learning in Sciencecited by 32
  9. 2021DKM: Differentiable K-Means Clustering Layer for Neural Network Compressioncited by 31
  10. 2021GDP: Stabilized Neural Network Pruning via Gates with Differentiable Polarizationcited by 29
  11. 2021An Information-Theoretic Justification for Model Pruningcited by 25
  12. 2023SlimSAM: 0.1% Data Makes Segment Anything Slimcited by 20
38 more
  1. 2021RED : Looking for Redundancies for Data-Free Structured Compression of Deep Neural Networkscited by 19
  2. 2021RED++ : Data-Free Pruning of Deep Neural Networks via Input Splitting and Output Mergingcited by 19
  3. 2026Agentic World Modeling: Foundations, Capabilities, Laws, and Beyondcited by 14
  4. 2022Is Complexity Required for Neural Network Pruning? A Case Study on Global Magnitude Pruningcited by 13
  5. 2024AMD: Automatic Multi-step Distillation of Large-scale Vision Modelscited by 12
  6. 2022MKQ-BERT: Quantized BERT with 4-bits Weights and Activationscited by 10
  7. 2023F3-Pruning: A Training-Free and Generalized Pruning Strategy towards Faster and Finer Text-to-Video Synthesiscited by 10
  8. 2022SInGE: Sparsity via Integrated Gradients Estimation of Neuron Relevancecited by 9
  9. 2025On-device edge learning for IoT data streams: a surveycited by 8
  10. 2021LEAP: Learnable Pruning for Transformer-based Modelscited by 7
  11. 2022Perturbation of Deep Autoencoder Weights for Model Compression and Classification of Tabular Datacited by 7
  12. 2024Data-independent Module-aware Pruning for Hierarchical Vision Transformerscited by 6
  13. 2023Utility-based Perturbed Gradient Descent: An Optimizer for Continual Learningcited by 5
  14. 2026FastLightGen: Fast and Light Video Generation with Fewer Steps and Parameterscited by 5
  15. 2021AdaptCL: Efficient Collaborative Learning with Dynamic and Adaptive Pruningcited by 4
  16. 2023eDKM: An Efficient and Accurate Train-time Weight Clustering for Large Language Modelscited by 4
  17. 2023PriPrune: Quantifying and Preserving Privacy in Pruned Federated Learningcited by 4
  18. 2025ALTER: All-in-One Layer Pruning and Temporal Expert Routing for Efficient Diffusion Generationcited by 4
  19. 2021Deep Neural Compression Via Concurrent Pruning and Self-Distillationcited by 3
  20. 2023Resource Efficient Neural Networks Using Hessian Based Pruningcited by 3
  21. 2024ONNXPruner: ONNX-Based General Model Pruning Adaptercited by 3
  22. 2026Structural Pruning of Large Vision Language Models: A Comprehensive Study on Pruning Dynamics, Recovery, and Data Efficiencycited by 3
  23. 2021ACP: Automatic Channel Pruning via Clustering and Swarm Intelligence Optimization for CNNcited by 2
  24. 2023Sparse Weight Averaging with Multiple Particles for Iterative Magnitude Pruningcited by 2
  25. 2024From Algorithm to Hardware: A Survey on Efficient and Safe Deployment of Deep Neural Networkscited by 2
  26. 2025Pruning-based Data Selection and Network Fusion for Efficient Deep Learningcited by 2
  27. 2026Towards Understanding Best Practices for Quantization of Vision-Language Modelscited by 2
  28. 2021Pruning with Compensation: Efficient Channel Pruning for Deep Convolutional Neural Networkscited by 1
  29. 2022Aligned Weight Regularizers for Pruning Pretrained Neural Networkscited by 1
  30. 2022ViNNPruner: Visual Interactive Pruning for Deep Learningcited by 1
  31. 2022Design Automation for Fast, Lightweight, and Effective Deep Learning Models: A Surveycited by 1
  32. 2023Layer-adaptive Structured Pruning Guided by Latencycited by 1
  33. 2024Towards Explaining Deep Neural Network Compression Through a Probabilistic Latent Spacecited by 1
  34. 2024Torch2Chip: An End-to-end Customizable Deep Neural Network Compression and Deployment Toolkit for Prototype Hardware Accelerator Designcited by 1
  35. 2024AutoSculpt: A Pattern-based Model Auto-pruning Framework Using Reinforcement Learning and Graph Learningcited by 1
  36. 2025Investigating Structural Pruning and Recovery Techniques for Compressing Multimodal Large Language Models: An Empirical Studycited by 1
  37. 2025EfficientXpert: Efficient Domain Adaptation for Large Language Models via Propagation-Aware Pruningcited by 1
  38. 2021Dep-L0: Improving L0-based Network Sparsification via Dependency Modelingcited by 0

50 most cited of 61 All 61 (JSON)