mlsanityData to 7 Oct 2026

PDTrim: Targeted Pruning for Prefill-Decode Disaggregation in Inference

arXiv 2509.04467 ↗First posted 29 Aug 2025cs.CL cs.AIcited by 14

On the map: Efficient LLM inference and serving → Early exit and edge inference

Abstract (arXiv)

Large Language Models (LLMs) demonstrate exceptional capabilities across various tasks, but their deployment is constrained by high computational and memory costs. Model pruning provides an effective means to alleviate these demands. However, existing methods often ignore the characteristics of prefill-decode (PD) disaggregation in practice. In this paper, we propose a pruning method that is highly integrated with PD disaggregation, enabling more precise pruning of blocks. Our approach constructs pruning and distillation sets to perform iterative block removal, obtaining better pruning solutions. Moreover, we analyze the pruning sensitivity of the prefill and decode stages and identify removable blocks specific to each stage, making it well suited for PD disaggregation deployment. Extensive experiments demonstrate our approach consistently achieves strong performance in both PD disaggregation and PD unified (non-PD disaggregation) settings, and can also be extended to other non-block pruning methods. Under the same settings, our method achieves improved performance and faster inference.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Early exit and edge inference, one of 12 directions in Efficient LLM inference and serving. Builds on 54 of its 64 references that are in the map. First posted on arXiv 29 Aug 2025.

Builds on

54 of 64 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024LazyLLM: Dynamic Token Pruning for Efficient Long Context LLM Inference19 Jul

Other references · 53

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  7. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  8. 2024Qwen2.5 Technical Reportcited by 4,713
  9. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  10. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  11. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  12. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
38 more
  1. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  2. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  3. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  4. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  5. 2020Knowledge Distillation: A Surveycited by 1,578
  6. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
  7. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  8. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
  9. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 893
  10. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 848
  11. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 805
  12. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 653
  13. 2023LLM-Pruner: On the Structural Pruning of Large Language Modelscited by 643
  14. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 595
  15. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 536
  16. 2019SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarizationcited by 530
  17. 2023CMMLU: Measuring massive multitask language understanding in Chinesecited by 436
  18. 2024SliceGPT: Compress Large Language Models by Deleting Rows and Columnscited by 378
  19. 2024MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attentioncited by 375
  20. 2024ShortGPT: Layers in Large Language Models are More Redundant Than You Expectcited by 369
  21. 2021Post-Training Quantization for Vision Transformercited by 274
  22. 2024InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Managementcited by 242
  23. 2024Mooncake: A KVCache-centric Disaggregated Architecture for LLM Servingcited by 193
  24. 2024Compact Language Models via Pruning and Knowledge Distillationcited by 142
  25. 2024LaCo: Large Language Model Pruning via Layer Collapsecited by 124
  26. 2024MiniCache: KV Cache Compression in Depth Dimension for Large Language Modelscited by 122
  27. 2025FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inferencecited by 121
  28. 2024Inference without Interference: Disaggregate LLM Inference for Mixed Downstream Workloadscited by 120
  29. 2023Fluctuation-based Adaptive Structured Pruning for Large Language Modelscited by 110
  30. 2024TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decodingcited by 98
  31. 2024SLEB: Streamlining LLMs through Redundancy Verification and Elimination of Transformer Blockscited by 96
  32. 2024Shortened LLaMA: Depth Pruning for Large Language Models with Comparison of Retraining Methodscited by 80
  33. 2024MemServe: Context Caching for Disaggregated LLM Serving with Elastic Memory Poolcited by 67
  34. 2024Streamlining Redundant Layers to Compress Large Language Modelscited by 32
  35. 2025Prefill-Decode Aggregation or Disaggregation? Unifying Both for Goodput-Optimized LLM Servingcited by 21
  36. 2024FrameQuant: Flexible Low-Bit Quantization for Transformerscited by 17
  37. 2024TQA-Bench: Evaluating LLMs for Multi-Table Question Answeringcited by 17
  38. 2025Injecting Adrenaline into LLM Serving: Boosting Resource Utilization and Throughput via Attention Disaggregationcited by 14

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 12.

14 papers.

Sort:

By area:

  1. 2026Multimodal Multi-Agent Empowered Legal Judgment Predictioncited by 14
  2. 2025TrimTokenator: Towards Adaptive Visual Token Pruning for Large Multimodal Modelscited by 9
  3. 2025HyperAdaLoRA: Accelerating LoRA Rank Allocation During Training via Hypernetworks without Sacrificing Performancecited by 9
  4. 2026ShieldedCode: Learning Robust Representations for Virtual Machine Protected Codecited by 8
  5. 2026FBS: Modeling Native Parallel Reading inside a Transformercited by 6
  6. 2026Perturbation Sensitivity of Maximum-Likelihood Pairwise Ranking in Computational Decision Systemscited by 1
  7. 2026MedEvoEval: Evaluating Continual Evolution of Doctor Agents through Simulated Clinical Episodescited by 0
  8. 2026FinInvest-GTCN: Explainable Graph-Temporal-Causal Modeling for Risk-Aware Investment Decision Optimizationcited by 0
  9. 2026EVLA: An Electro-Aware Multimodal Assistant for Physically-Grounded Driving Reasoning and Controlcited by 0
  10. 2026A3M: Adaptive, Adversarial and Multi-Objective Learning for Strategic Bidding in Repeated Auctionscited by 0
  11. 2026Dependency-Guided Code Generation: Structured Matrix Decomposition and Consistency-Guided Refinementcited by 0
  12. 2026Though Language Models Err While They Strive: Conformal Prediction for Self-Correcting Scientific Generationcited by 0
2 more
  1. 2026Code Consistency Preference Optimization Verification for Language Model Alignmentcited by 0
  2. 2026CompileRover: Revolutionizing Virtual Machine Compiler Optimization with a Tri-Role LLM-Driven Frameworkcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026MI-PRUN: Optimize Large Language Model Pruning via Mutual Information
  2. 2026POP: Prefill-Only Pruning for Efficient Large Model Inference
  3. 2026Beyond FLOPs: Benchmarking Real Inference Acceleration of LLM Pruning under a GEMM-Centric Taxonomy
  4. 2026SparseDecoding: Decoding-Aware Pruning for Accurate and Efficient LLM Inference
  5. 2024Rethinking Pruning Large Language Models: Benefits and Pitfalls of Reconstruction Error Minimization
  6. 2025SPAP: Structured Pruning via Alternating Optimization and Penalty Methods
  7. 2025Pruning Laws for Large Language Models
  8. 2025Accurate Sublayer Pruning for Large Language Models by Exploiting Latency and Tunability Information