mlsanityData to 7 Oct 2026

Leveraging the true depth of LLMs

arXiv 2502.02790 ↗First posted 5 Feb 2025cs.LG cs.CLcited by 6

Not on the map. Why?

Abstract (arXiv)

The remarkable capabilities of Large Language Models (LLMs) are overshadowed by their immense computational cost. While recent work has shown that many LLM layers can be reordered or even removed with minimal impact on accuracy, these insights have not been translated into significant inference speedups. To bridge this gap, we introduce a novel method that restructures the computational graph by grouping and evaluating consecutive layer pairs in parallel. This approach, requiring no retraining, yields a 1.19x throughput gain on Llama 2 7B while reducing the average benchmark accuracy by only 1.5%. We demonstrate the practical value of this method for large-scale LLM deployment and show that some of the lost accuracy can be recovered with lightweight fine-tuning of the parallelized layers.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 38 of its 52 references that are in the map. First posted on arXiv 5 Feb 2025.

Builds on

38 of 52 references.

References · 38

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2014Going Deeper with Convolutionscited by 4,231
  7. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  8. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  9. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  10. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  11. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  12. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
26 more
  1. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  2. 2019Similarity of Neural Network Representations Revisitedcited by 1,695
  3. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  4. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
  5. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,090
  6. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 821
  7. 2021Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LMcited by 621
  8. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 591
  9. 2024ShortGPT: Layers in Large Language Models are More Redundant Than You Expectcited by 369
  10. 2019Q-BERT: Hessian Based Ultra Low Precision Quantization of BERTcited by 355
  11. 2020BERT Loses Patience: Fast and Robust Inference with Early Exitcited by 267
  12. 2015Highway Networkscited by 257
  13. 2020SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruningcited by 228
  14. 2024The Unreasonable Ineffectiveness of the Deeper Layerscited by 219
  15. 2024The Remarkable Robustness of LLMs: Stages of Inference?cited by 138
  16. 2024AI and Memory Wallcited by 89
  17. 2023A Mechanistic Interpretation of Arithmetic Reasoning in Language Models using Causal Mediation Analysiscited by 88
  18. 2024A Survey of Resource-efficient LLM and Multimodal Foundation Modelscited by 84
  19. 2020SpArch: Efficient Architecture for Sparse Matrix Multiplicationcited by 47
  20. 2024Interpreting Arithmetic Mechanism in Large Language Models through Comparative Neuron Analysiscited by 31
  21. 2024A Survey of Sustainability in Large Language Models: Applications, Economics, and Challengescited by 18
  22. 2024Math Neurosurgery: Isolating Language Models' Math Reasoning Abilities Using Only Forward Passescited by 10
  23. 2024TPI-LLM: Serving 70B-scale LLMs Efficiently on Low-resource Edge Devicescited by 9
  24. 2024Towards Low-bit Communication for Tensor Parallel LLM Inferencecited by 6
  25. 2025StagFormer: Time Staggering Transformer Decoding for RunningLayers In Parallelcited by 2
  26. 2019How Compact?: Assessing Compactness of Representations through Layer-Wise Pruningcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 3.

6 papers.

  1. 2025Learning to Skip the Middle Layers of Transformerscited by 8
  2. 2025Predictability Enables Parallelization of Nonlinear State Space Modelscited by 7
  3. 2026Reclaiming Lost Text Layers for Source-Free Cross-Domain Few-Shot Learningcited by 5
  4. 2026Modularity is the Bedrock of Natural and Artificial Intelligencecited by 2
  5. 2025Statistical Deficiency for Task Inclusion Estimationcited by 1
  6. 2026SNLP: Layer-Parallel Inference via Structured Newton Correctionscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2023EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D Parallelism
  2. 2024LLM Inference Unveiled: Survey and Roofline Model Insights
  3. 2023Efficient LLM Inference on CPUs
  4. 2024A Survey on Efficient Inference for Large Language Models
  5. 2024SLEB: Streamlining LLMs through Redundancy Verification and Elimination of Transformer Blocks
  6. 2024GUIDE: A Global Unified Inference Engine for Deploying Large Language Models in Heterogeneous Environments
  7. 2025Seesaw: High-throughput LLM Inference via Model Re-sharding
  8. 2025Taming the Titans: A Survey of Efficient LLM Inference Serving