mlsanityData to 14 Oct 2026

DeltaZip: Efficient Serving of Multiple Full-Model-Tuned LLMs

arXiv 2312.05215 ↗First posted 8 Dec 2023cs.DC cs.LGcited by 18

Not on the map. Why?

Abstract (arXiv)

Fine-tuning large language models (LLMs) greatly improves model quality for downstream tasks. However, serving many fine-tuned LLMs concurrently is challenging due to the sporadic, bursty, and varying request patterns of different LLMs. To bridge this gap, we present DeltaZip, an LLM serving system that efficiently serves multiple full-parameter fine-tuned models concurrently by aggressively compressing model deltas by up to 10x while maintaining high model quality. The key insight behind this design is that fine-tuning results in small-magnitude changes to the pre-trained model. By co-designing the serving system with the compression algorithm, DeltaZip achieves 2x to 12x improvement in throughput compared to the state-of-the-art systems.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Builds on 29 of its 82 references that are in the map. First posted on arXiv 8 Dec 2023.

Builds on

29 of 82 references.

References · 29

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  5. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  6. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  7. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  8. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  9. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  10. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  11. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,475
  12. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
17 more
  1. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,125
  2. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
  3. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 839
  4. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 699
  5. 2023Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunchcited by 694
  6. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 558
  7. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 526
  8. 2016Clipper: A Low-Latency Online Prediction Serving Systemcited by 191
  9. 2021Learning N:M Fine-grained Structured Sparse Neural Networks From Scratchcited by 179
  10. 2020Serving DNNs like Clockwork: Performance Predictability from the Bottom Upcited by 120
  11. 2021Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Maskscited by 103
  12. 2023Goat: Fine-tuned LLaMA Outperforms GPT-4 on Arithmetic Taskscited by 84
  13. 2023Punica: Multi-Tenant LoRA Servingcited by 83
  14. 2020Serverless in the Wild: Characterizing and Optimizing the Serverless Workload at a Large Cloud Providercited by 66
  15. 2023Do Emergent Abilities Exist in Quantized Large Language Models: An Empirical Studycited by 43
  16. 2019ViGGO: A Video Game Corpus for Data-To-Text Generation in Open-Domain Conversationcited by 31
  17. 2023INT2.1: Towards Fine-Tunable Quantized Large Language Models with Error Correction through Low-Rank Adaptationcited by 20

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 5; 2026: 10.

18 papers.

Sort:

By area:

  1. 2024BitDelta: Your Fine-Tune May Only Be Worth One Bitcited by 39
  2. 2025RLBoost: Harvesting Preemptible Resources for Cost-Efficient Reinforcement Learning on LLMscited by 11
  3. 2025TokenFlow: Responsive LLM Text Streaming Serving under Request Burst via Preemptive Schedulingcited by 10
  4. 2025ImPart: Importance-Aware Delta-Sparsification for Improved Model Compression and Merging in LLMscited by 5
  5. 2024DeltaDQ: Ultra-High Delta Compression for Fine-Tuned LLMs via Group-wise Dropout and Separate Quantizationcited by 3
  6. 2024ME-Switch: A Memory-Efficient Expert Switching Framework for Large Language Modelscited by 2
  7. 2026ForkKV: Scaling Multi-LoRA Agent Serving via Copy-on-Write Disaggregated KV Cachecited by 2
  8. 2026SUN: Shared Use of Next-token Prediction for Efficient Multi-LLM Disaggregated Servingcited by 1
  9. 2026D-QRELO: Training- and Data-Free Delta Compression for Large Language Models via Quantization and Residual Low-Rank Approximationcited by 1
  10. 2025SineLoRADelta: Sine-Activated Delta Compressioncited by 0
  11. 2025Enhancing Delta Compression in LLMs via SVD-based Quantization Error Minimizationcited by 0
  12. 2026LegoDiffusion: Micro-Serving Text-to-Image Diffusion Workflowscited by 0
6 more
  1. 2026SOLAR: Communication-Efficient Model Adaptation via Subspace-Oriented Latent Adapter Reparametrizationcited by 0
  2. 2026TensorDex: A Compact, Tensor-Centric Storage System for Modern AI Modelscited by 0
  3. 2026ServerlessT2I: Efficient Text-to-Image Workflow Serving on a Serverless Platformcited by 0
  4. 2026Simthesizer: An Agent-Driven Simulation Framework for LLM Serving Systemscited by 0
  5. 2026Block-Level Weight-Space Structure Persists Under Post-Training: An Empirical Study Across LLM Familiescited by 0
  6. 2026JointServe: Serving Multi-Model Agents on Memory-Constrained GPUscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Delta-CoMe: Training-Free Delta-Compression with Mixed-Precision for Large Language Models
  2. 2024ScaleLLM: A Resource-Frugal LLM Serving Framework by Optimizing End-to-End Efficiency
  3. 2023Dissecting the Runtime Performance of the Training, Fine-tuning, and Inference of Large Language Models
  4. 2025From Large to Super-Tiny: End-to-End Optimization for Cost-Efficient LLMs
  5. 2026AE-LLM: Adaptive Efficiency Optimization for Large Language Models
  6. 2024The CAP Principle for LLM Serving: A Survey of Long-Context Large Language Model Serving
  7. 2025Optimizing LLMs for Resource-Constrained Environments: A Survey of Model Compression Techniques
  8. 2025DP-LLM: Runtime Model Adaptation with Dynamic Layer-wise Precision Assignment