mlsanityData to 14 Oct 2026

70% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float (DFloat11)

arXiv 2504.11651 ↗First posted 15 Apr 2025cs.LG cs.DCcited by 24

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Large-scale AI models, such as Large Language Models (LLMs) and Diffusion Models (DMs), have grown rapidly in size, creating significant challenges for efficient deployment on resource-constrained hardware. In this paper, we introduce Dynamic-Length Float (DFloat11), a lossless compression framework that reduces LLM and DM size by 30% while preserving outputs that are bit-for-bit identical to the original model. DFloat11 is motivated by the low entropy in the BFloat16 weight representation of LLMs, which reveals significant inefficiency in the existing storage format. By applying entropy coding, DFloat11 assigns dynamic-length encodings to weights based on frequency, achieving near information-optimal compression without any loss of precision. To facilitate efficient inference with dynamic-length encodings, we develop a custom GPU kernel for fast online decompression. Our design incorporates the following: (i) compact, hierarchical lookup tables (LUTs) that fit within GPU SRAM for efficient decoding, (ii) a two-phase GPU kernel for coordinating thread read/write positions using lightweight auxiliary variables, and (iii) transformer-block-level decompression to minimize latency. Experiments on Llama 3.3, Qwen 3, Mistral 3, FLUX.1, and others validate our hypothesis that DFloat11 achieves around 30% model size reduction while preserving bit-for-bit identical outputs. Compared to a potential alternative of offloading parts of an uncompressed model to the CPU to meet memory constraints, DFloat11 achieves 2.3--46.2x higher throughput in token generation. With a fixed GPU memory budget, DFloat11 enables 5.7--14.9x longer generation lengths than uncompressed models. Notably, our method enables lossless inference of Llama 3.1 405B, an 810GB model, on a single node equipped with 8x80GB GPUs.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 50 of its 69 references that are in the map. First posted on arXiv 15 Apr 2025.

Builds on

50 of 69 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024BitsFusion: 1.99 bits Weight Quantization of Diffusion Model6 Jun
  2. 2023Q-Diffusion: Quantizing Diffusion Models8 Feb
  3. 2022Diffusion Models: A Comprehensive Survey of Methods and Applications2 Sep

Other references · 47

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  6. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  7. 2024Qwen2.5 Technical Reportcited by 4,942
  8. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  9. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  10. 2016Pointer Sentinel Mixture Modelscited by 3,096
  11. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  12. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
35 more
  1. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  2. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  3. 2025Gemma 3 Technical Reportcited by 1,824
  4. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  5. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  6. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  7. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
  8. 2024Phi-4 Technical Reportcited by 711
  9. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 637
  10. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 591
  11. 2024GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projectioncited by 463
  12. 2023Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyondcited by 411
  13. 2024Break the Sequential Dependency of LLM Inference Using Lookahead Decodingcited by 298
  14. 2024Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decodingcited by 269
  15. 2019A Study of BFLOAT16 for Deep Learning Trainingcited by 232
  16. 2022Speculative Decoding: Exploiting Speculative Execution for Accelerating Seq2seq Generationcited by 171
  17. 2024A Survey of Resource-efficient LLM and Multimodal Foundation Modelscited by 86
  18. 2024Evaluating Quantized Large Language Modelscited by 77
  19. 2024KV Cache is 1 Bit Per Channel: Efficient Large Language Model Inference with Coupled Quantizationcited by 76
  20. 2025Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Modelscited by 73
  21. 2024A Comprehensive Evaluation of Quantization Strategies for Large Language Modelscited by 48
  22. 2024LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkitcited by 33
  23. 2024Accuracy is Not All You Needcited by 31
  24. 2024LeanQuant: Accurate and Scalable Large Language Model Quantization with Loss-error-aware Gridcited by 22
  25. 2024Learning from Students: Applying t-Distributions to Explore Accurate and Efficient Formats for LLMscited by 18
  26. 2025Investigating the Impact of Quantization Methods on the Safety and Reliability of Large Language Modelscited by 18
  27. 2024LLMCBench: Benchmarking Large Language Model Compression for Efficient Deploymentcited by 16
  28. 2024NeuZip: Memory-Efficient Training and Inference with Dynamic Compression of Neural Networkscited by 14
  29. 2024ZipNN: Lossless Compression for AI Modelscited by 14
  30. 2024NoMAD-Attention: Efficient LLM Inference on CPUs Through Multiply-add-free Attentioncited by 12
  31. 2023NF4 Isn't Information Theoretically Optimal (and that's Good)cited by 11
  32. 2025Huff-LLM: End-to-End Lossless Compression for Efficient LLM Inferencecited by 9
  33. 2025Breaking the Frozen Subspace: Importance Sampling for Low-Rank Optimization in LLM Pretrainingcited by 7
  34. 2024Sketch to Adapt: Fine-Tunable Sketches for Efficient LLM Adaptationcited by 3
  35. 2024Balancing Speed and Stability: The Trade-offs of FP8 vs. BF16 Training in LLMscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 8; 2026: 16.

24 papers.

Sort:

By area:

  1. 2025Get Experience from Practice: LLM Agents with Record & Replaycited by 23
  2. 2025Don't Pass@k: A Bayesian Framework for Large Language Model Evaluationcited by 15
  3. 2025Demystifying Hybrid Thinking: Can LLMs Truly Switch Between Think and No-Think?cited by 9
  4. 2025PuzzleMoE: Efficient Compression of Large Mixture-of-Experts Models via Sparse Expert Merging and Bit-packed inferencecited by 9
  5. 2026ZipServ: Fast and Memory-Efficient LLM Inference with Hardware-Aware Lossless Compressioncited by 8
  6. 2025Neural Network Quantization for Microcontrollers: A Comprehensive Survey of Methods, Platforms, and Applicationscited by 7
  7. 2026KVServe: Service-Aware KV Cache Compression for Communication-Efficient Disaggregated LLM Servingcited by 7
  8. 2025Nightjar: Dynamic Adaptive Speculative Decoding for Large Language Models Servingcited by 6
  9. 2026Statistically-Lossless Quantization of Large Language Modelscited by 6
  10. 2025To Compress or Not? Pushing the Frontier of Lossless GenAI Model Weights Compression with Exponent Concentrationcited by 5
  11. 2026ZipMoE: Efficient On-Device MoE Serving via Lossless Compression and Cache-Affinity Schedulingcited by 4
  12. 2026ENEC: A Lossless AI Model Compression Method Enabling Fast Inference on Ascend NPUscited by 3
12 more
  1. 2026Path-Lock Expert: Separating Reasoning Mode in Hybrid Thinking via Architecture-Level Separationcited by 3
  2. 2026SplitZip: Ultra Fast Lossless KV Compression for Disaggregated LLM Servingcited by 3
  3. 2026Float8@2bits: Entropy Coding Enables Data-Free Model Compressioncited by 2
  4. 2025Optimal Formats for Weight Quantisationcited by 1
  5. 2026Ranking Reasoning LLMs under Test-Time Scalingcited by 1
  6. 2026ZipCCL: Efficient Lossless Data Compression of Communication Collectives for Accelerating LLM Trainingcited by 1
  7. 2026Embedding Compression via Spherical Coordinatescited by 0
  8. 2026FluxMoE: Decoupling Expert Residency for High-Performance MoE Servingcited by 0
  9. 2026Cassandra: Enabling Reasoning LLMs at Edge via Self-Speculative Decodingcited by 0
  10. 2026Reducing the GPU Memory Bottleneck with Lossless Compression for ML -- Extendedcited by 0
  11. 2026EntroPack: Fast and Accurate Entropy-Coded Weight Compression at Arbitrary Bitratescited by 0
  12. 2026NeuralZip: Reusable Setup for Fast Lossless Compressioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026The Economics of AI Decoding Chips: Rebalancing Compute, Capacity, and Bandwidth for Efficient LLM Inference
  2. 2026Budget-Aware Compression Pipeline for Single-GPU LLM Inference: Methods, Trade-offs, and Coupling Effects
  3. 2023Efficient LLM inference solution on Intel GPU
  4. 2023Fast and Efficient 2-bit LLM Inference on GPU: 2/4/16-bit in a Weight Matrix with Asynchronous Dequantization
  5. 2025SpecOffload: Unlocking Latent GPU Capacity for LLM Inference on Resource-Constrained Devices
  6. 2025Characterizing LLM Inference Energy-Performance Tradeoffs across Workloads and GPU Scaling
  7. 2024Dynamic Memory Compression: Retrofitting LLMs for Accelerated Inference
  8. 2025Reimagining Memory Access for LLM Inference: Compression-Aware Memory Controller Design