# EBFT: Effective and Block-Wise Fine-Tuning for Sparse LLMs

arXiv: https://arxiv.org/abs/2402.12419 · HTML: https://mlsanity.com/paper/2402.12419/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Feb 2024.

## Built on by (most cited first, 6 of 6)

- [Dynamic Low-Rank Sparse Adaptation for Large Language Models](https://mlsanity.com/paper/2502.14816/) (2025)
- [Rethinking Pruning Large Language Models: Benefits and Pitfalls of Reconstruction Error Minimization](https://mlsanity.com/paper/2406.15524/) (2024)
- [Zeroth-Order Adaptive Neuron Alignment Based Pruning without Re-Training](https://mlsanity.com/paper/2411.07066/) (2024)
- [An Efficient Sparse Fine-Tuning with Low Quantization Error via Neural Network Pruning](https://mlsanity.com/paper/2502.11439/) (2025)
- [From 2:4 to 8:16 sparsity patterns in LLMs for Outliers and Weights with Variance Correction](https://mlsanity.com/paper/2507.03052/) (2025)
- [A Free Lunch in LLM Compression: Revisiting Retraining after Pruning](https://mlsanity.com/paper/2510.14444/) (2025)

## Abstract (arXiv)

Existing methods for fine-tuning sparse LLMs often suffer from resource-intensive requirements and high retraining costs. Additionally, many fine-tuning methods often rely on approximations or heuristic optimization strategies, which may lead to suboptimal solutions. To address these issues, we propose an efficient and fast framework for fine-tuning sparse LLMs based on minimizing reconstruction error. Our approach involves sampling a small dataset for calibration and utilizing backpropagation to iteratively optimize block-wise reconstruction error, on a block-by-block basis, aiming for optimal solutions. Extensive experiments on various benchmarks consistently demonstrate the superiority of our method over other baselines. For instance, on the Wikitext2 dataset with LlamaV1-7B at 70% sparsity, our proposed EBFT achieves a perplexity of 16.88, surpassing the state-of-the-art DSnoT with a perplexity of 75.14. Moreover, with a structured sparsity ratio of 26%, EBFT achieves a perplexity of 16.27, outperforming LoRA (perplexity 16.44). Furthermore, the fine-tuning process of EBFT for LlamaV1-7B only takes approximately 30 minutes, and the entire framework can be executed on a single 16GB GPU. The source code is available at https://github.com/sunggo/EBFT.