mlsanityData to 7 Oct 2026

From Layers to Submodules: Rethinking Granularity in Replacement-Based LLM Compression

arXiv 2606.02559 ↗First posted 1 Jun 2026cs.CL cs.AIcited by 1

On the map: LLM reasoning → Test-time compute and efficient thinking

also in 1 more
Abstract (arXiv)

Post-training compression of Large Language Models (LLMs) removes entire architectural components, either deleting them or replacing them with fitted modules. Existing replacement-based methods share two design constraints: full-layer granularity and contiguous selection. We argue that this is overly restrictive: in fact, redundancy in pretrained transformers is not confined to contiguous regions, nor does it evenly distribute between Attention and FeedForward outputs, implying that different strategies best approximate different submodule types and that removable components need not cluster within contiguous depth ranges. Based on this intuition, we introduce SubFit (Submodule-level Fitted residual replacement), which compresses LLMs at the submodule level: Attention and FeedForward submodules are selected non-contiguously, and each receives its own lightweight fitted residual bypass. SubFit operates post-training and requires only calibration data. Across ten LLMs (five base, five instruction-tuned), five sparsity levels from 12.5% to 37.5%, and four replacement-based baselines, SubFit achieves the best aggregate perplexity-accuracy trade-off across the evaluated sparsity levels, with larger gains under aggressive compression. At 25% sparsity, it retains 84.6% of dense downstream accuracy and incurs 2.42x perplexity degradation, against 81.6% and 4.34x for the strongest baselines, while delivering measurable inference speedup and KV-cache savings. Code is available at https://github.com/eliacunegatti/SubFit.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 17 of its 60 references that are in the map. First posted on arXiv 1 Jun 2026.

Builds on

17 of 60 references.

Closest ancestors · 1

Same direction, newest first

  1. 2026On the Limits of Layer Pruning for Generative Reasoning in Large Language Models2 Feb

Other references · 16

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2025Qwen3 Technical Reportcited by 8,195
  5. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  6. 2016Pointer Sentinel Mixture Modelscited by 2,992
  7. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  8. 2020GLU Variants Improve Transformercited by 1,724
  9. 2024Qwen2.5-Coder Technical Reportcited by 1,151
  10. 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermismcited by 621
  11. 2023Orca: Progressive Learning from Complex Explanation Traces of GPT-4cited by 369
  12. 2024What Matters in Transformers? Not All Attention is Neededcited by 83
4 more
  1. 2023SlimPajama-DC: Understanding Data Combinations for LLM Trainingcited by 72
  2. 2025DarwinLM: Evolutionary Structured Pruning of Large Language Modelscited by 16
  3. 2025Massive Supervised Fine-tuning Experiments Reveal How Data, Layer, and Training Factors Shape LLM Alignment Qualitycited by 14
  4. 2025DeltaLLM: Compress LLMs with Low-Rank Deltas between Shared Weightscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026DepthWeave-KV: Token-Adaptive Cross-Layer Residual Factorization for Long-Context KV Cache Compressioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Compressing LLMs: The Truth is Rarely Pure and Never Simple
  2. 2026Rethinking the Role of Tensor Decompositions in Post-Training LLM Compression
  3. 2024Streamlining Redundant Layers to Compress Large Language Models
  4. 2023Rethinking Compression: Reduced Order Modelling of Latent Features in Large Language Models
  5. 2025Compression is Routing: Reconstruction Error as an Intrinsic Signal for Modular Language Models
  6. 2026Compressed-Sensing-Guided, Inference-Aware Structured Reduction for Large Language Models
  7. 2026Capability-Guided Compression: Toward Interpretability-Aware Budget Allocation for Large Language Models
  8. 2024Compressing Large Language Models with Automated Sub-Network Search