mlsanityData to 14 Oct 2026

HyperZip: Efficient Data Compression through Personalized Diffusion LLMs with Hypernetworks

arXiv 2609.36357 ↗First posted 28 Sep 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have shown strong potential for lossless data compression, but existing approaches are constrained by the high computational cost and low throughput of autoregressive decoding. We propose HyperZip, an efficient and scalable LLM-based compression framework that leverages diffusion-based LLMs (dLLMs) with Multi-Token Prediction (MTP) to accelerate LLM-based data compression processes. We identify a trade-off in diffusion-based compression, where increasing decoding throughput degrades the compression rate. To mitigate this trade-off, HyperZip employs a hypernetwork to generate data-specific updates from a context representation, adapting the dLLM to the target data without costly fine-tuning, resulting in a low compression rate and high throughput. Extensive experiments show that HyperZip achieves a superior trade-off between compression rate and speed compared with state-of-the-art baselines.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 47 of its 66 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

47 of 66 references.

References · 47

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2024Qwen2.5 Technical Reportcited by 4,942
  4. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  5. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  6. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,160
  7. 2016HyperNetworkscited by 1,027
  8. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 1,006
  9. 2025Large Language Diffusion Modelscited by 912
  10. 2023A Comprehensive Overview of Large Language Modelscited by 648
  11. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 626
  12. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 574
35 more
  1. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Modelscited by 409
  2. 2024Better & Faster Large Language Models via Multi-token Predictioncited by 360
  3. 2025SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Modelcited by 335
  4. 2025EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Testcited by 331
  5. 2023Language Modeling Is Compressioncited by 225
  6. 2025Fast-dLLM v2: Efficient Block-Diffusion LLMcited by 139
  7. 2026DFlash: Block Diffusion for Flash Speculative Decodingcited by 135
  8. 2023A Brief Review of Hypernetworks in Deep Learningcited by 102
  9. 2022An Introduction to Neural Data Compressioncited by 79
  10. 2025Diffusion Language Models Know the Answer Before Decodingcited by 70
  11. 2025Text-to-LoRA: Instant Transformer Adaptioncited by 58
  12. 2026Doc-to-LoRA: Learning to Instantly Internalize Contextscited by 43
  13. 2025Efficient-DLM: From Autoregressive to Diffusion Language Models, and Beyond in Speedcited by 33
  14. 2024Recent Advances in Generative AI and Large Language Models: Current Status, Challenges, and Perspectivescited by 30
  15. 2025A Survey on Parallel Text Generation: From Parallel Decoding to Diffusion Language Modelscited by 30
  16. 2026Nemotron-Labs-Diffusion: A Tri-Mode Language Model Unifying Autoregressive, Diffusion, and Self-Speculation Decodingcited by 24
  17. 2025Speculative Decoding and Beyond: An In-Depth Survey of Techniquescited by 23
  18. 2025From Bits to Rounds: Parallel Decoding with Exploration for Diffusion Language Modelscited by 23
  19. 2022Hyperdecoders: Instance-specific decoders for multi-task NLPcited by 22
  20. 2024Lossless data compression by large modelscited by 20
  21. 2025Sequential Diffusion Language Modelscited by 19
  22. 2025Instant Personalized Large Language Model Adaptation via Hypernetworkcited by 18
  23. 2026Confidence-Based Decoding is Provably Efficient for Diffusion Language Modelscited by 18
  24. 2025Understanding LLM Behaviors via Compression: Data Generation, Knowledge Acquisition and Scaling Lawscited by 15
  25. 2025L-MTP: Leap Multi-Token Prediction Beyond Adjacent Context for Large Language Modelscited by 15
  26. 2024Compression via Pre-trained Transformers: A Study on Byte-Level Multimodal Datacited by 13
  27. 2025Fast-Decoding Diffusion Language Models via Progress-Aware Confidence Schedulescited by 11
  28. 2024Large Language Models for Lossless Image Compression: Next-Pixel Prediction in Language Space is All You Needcited by 10
  29. 2024L3TC: Leveraging RWKV for Learned Lossless Low-Complexity Text Compressioncited by 7
  30. 2026Multi-Token Prediction via Self-Distillationcited by 7
  31. 2025Test-Time Steering for Lossless Text Compression via Weighted Product of Expertscited by 3
  32. 2026OmniZip: Learning a Unified and Lightweight Lossless Compressor for Multi-Modal Datacited by 2
  33. 2026P-MTP: Efficient Document Parsing via Multi-Token Prediction with Progressive Depth Scalingcited by 2
  34. 2026Scaling Laws for Hypernetwork-Based Knowledge Injection in Large Language Modelscited by 2
  35. 2026Adapting Diffusion Language Models for Lossless Pixel-Level Image Transmissioncited by 1

Built on by

Not cited yet.