mlsanityData to 14 Oct 2026

Prompt-based Depth Pruning of Large Language Models

arXiv 2502.04348 ↗First posted 4 Feb 2025cs.CL cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Depth pruning aims to reduce the inference cost of a large language model without any hardware-specific complications, by simply removing several less important transformer blocks. However, our empirical findings suggest that the importance of a transformer block may be highly task-dependent -- a block that is crucial for a task can be removed without degrading the accuracy on another task. Based on this observation, we develop a dynamic depth pruning algorithm, coined PuDDing (Prompt-routed Dynamic Depth Pruning), which determines which blocks to omit from the model based on the input prompt. PuDDing operates by training a lightweight router to predict the best omission set among a set of options, where this option set has also been constructed in a data-driven manner. Empirical results on commonsense reasoning benchmarks demonstrate that PuDDing effectively accelerates the inference language models, and achieves better on-task performance than static depth pruning baselines.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 25 of its 33 references that are in the map. First posted on arXiv 4 Feb 2025.

Builds on

25 of 33 references.

References · 25

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  5. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  6. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  7. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  8. 2016Pointer Sentinel Mixture Modelscited by 3,096
  9. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  10. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  11. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
  12. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,164
13 more
  1. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  2. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,043
  3. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 662
  4. 2024SliceGPT: Compress Large Language Models by Deleting Rows and Columnscited by 385
  5. 2024ShortGPT: Layers in Large Language Models are More Redundant Than You Expectcited by 383
  6. 2024Mixture-of-Depths: Dynamically allocating compute in transformer-based language modelscited by 258
  7. 2023Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Timecited by 252
  8. 2023Fluctuation-based Adaptive Structured Pruning for Large Language Modelscited by 111
  9. 2024SLEB: Streamlining LLMs through Redundancy Verification and Elimination of Transformer Blockscited by 97
  10. 2024What Matters in Transformers? Not All Attention is Neededcited by 84
  11. 2024CATS: Contextually-Aware Thresholding for Sparsity in Large Language Modelscited by 41
  12. 2024A deeper look at depth pruning of LLMscited by 23
  13. 2024ShadowLLM: Predictor-based Contextual Sparsity for Large Language Modelscited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Rethinking Layer Redundancy: Calibration Matters More Than Search in LLM Depth Pruningcited by 2
  2. 2026BUDDY: BUdget-Driven DYnamic Depth Routing for Adaptive Large Language Model Inferencecited by 1
  3. 2026Unifying Depth and Width Pruning for LLMs via Binary Knapsack Optimizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond FLOPs: Benchmarking Real Inference Acceleration of LLM Pruning under a GEMM-Centric Taxonomy
  2. 2025Pruning Laws for Large Language Models
  3. 2026POP: Prefill-Only Pruning for Efficient Large Model Inference
  4. 2024Pruning All-Rounder: Rethinking and Improving Inference Efficiency for Large Vision Language Models
  5. 2024FinerCut: Finer-grained Interpretable Layer Pruning for Large Language Models
  6. 2026Unlocking Structured Pruning for Reasoning Language Models via Attention-Preserving MLP Trimming
  7. 2025EfficientLLaVA:Generalizable Auto-Pruning for Large Vision-language Models
  8. 2025Automatic Pruning Discovery for Large Language Models