mlsanityData to 14 Oct 2026

NeuZip: Memory-Efficient Training and Inference with Dynamic Compression of Neural Networks

arXiv 2410.20650 ↗First posted 28 Oct 2024cs.LG cs.AI stat.MLcited by 14

Not on the map. Why?

Abstract (arXiv)

The performance of neural networks improves when more parameters are used. However, the model sizes are constrained by the available on-device memory during training and inference. Although applying techniques like quantization can alleviate the constraint, they suffer from performance degradation. In this work, we introduce NeuZip, a new weight compression scheme based on the entropy of floating-point numbers in neural networks. With NeuZip, we are able to achieve memory-efficient training and inference without sacrificing performance. Notably, we significantly reduce the memory footprint of training a Llama-3 8B model from 31GB to less than 16GB, while keeping the training dynamics fully unchanged. In inference, our method can reduce memory usage by more than half while maintaining near-lossless performance. Our code is publicly available.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 34 of its 45 references that are in the map. First posted on arXiv 28 Oct 2024.

Builds on

34 of 45 references.

References · 34

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  7. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  8. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  9. 2020Scaling Laws for Neural Language Modelscited by 6,586
  10. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  11. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
  12. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
22 more
  1. 2016Pointer Sentinel Mixture Modelscited by 3,096
  2. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  3. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  4. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  5. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  6. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,301
  7. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
  8. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,245
  9. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,139
  10. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 905
  11. 2016Training Deep Nets with Sublinear Memory Costcited by 848
  12. 2024Yi: Open Foundation Models by 01.AIcited by 719
  13. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 687
  14. 2017The Implicit Bias of Gradient Descent on Separable Datacited by 655
  15. 2024GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projectioncited by 463
  16. 2022The case for 4-bit precision: k-bit Inference Scaling Lawscited by 251
  17. 2019A Study of BFLOAT16 for Deep Learning Trainingcited by 232
  18. 2022A Fast Post-Training Pruning Framework for Transformerscited by 162
  19. 2020Exploring Versatile Generative Language Model Via Parameter-Efficient Transfer Learningcited by 139
  20. 2023Full Parameter Fine-tuning for Large Language Models with Limited Resourcescited by 134
  21. 2024Flora: Low-Rank Adapters Are Secretly Gradient Compressorscited by 108
  22. 2020Implicit Regularization in ReLU Networks with the Square Losscited by 45

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 11.

14 papers.

Sort:

By area:

  1. 202570% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float (DFloat11)cited by 24
  2. 2026ZipServ: Fast and Memory-Efficient LLM Inference with Hardware-Aware Lossless Compressioncited by 8
  3. 2025Scalable Parameter and Memory Efficient Pretraining for LLM: Recent Algorithmic Advances and Benchmarkingcited by 5
  4. 2026ZipMoE: Efficient On-Device MoE Serving via Lossless Compression and Cache-Affinity Schedulingcited by 4
  5. 2026Float8@2bits: Entropy Coding Enables Data-Free Model Compressioncited by 2
  6. 2026UCCL-Zip: Lossless Compression Supercharged GPU Communicationcited by 2
  7. 2026ZipCCL: Efficient Lossless Data Compression of Communication Collectives for Accelerating LLM Trainingcited by 1
  8. 2026Lossless Tensor Compression as Program Synthesiscited by 1
  9. 2025Reducing Storage of Pretrained Neural Networks by Rate-Constrained Quantization and Entropy Codingcited by 0
  10. 2026The Thermodynamic Costs of Simple Linear Regressioncited by 0
  11. 2026Transforms for LLM Quantization: The Great Inversion and Format Co-Designcited by 0
  12. 2026CIERA: Cross-Iteration Exponent Reuse for Lossless Allgather in Sharded MoE Trainingcited by 0
2 more
  1. 2026EntroPack: Fast and Accurate Entropy-Coded Weight Compression at Arbitrary Bitratescited by 0
  2. 2026NeuralZip: Reusable Setup for Fast Lossless Compressioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025EntroLLM: Entropy Encoded Weight Compression for Efficient Large Language Model Inference on Edge Devices
  2. 2023Bandwidth-efficient Inference for Neural Image Compression
  3. 2026Kilobyte Models: Neural Networks as a Seed and a Quantized Latent
  4. 2023End-to-End Neural Network Compression via fracℓ1ℓ2 Regularized Latency Surrogates
  5. 2026FlashOptim: Optimizers for Memory-Efficient Training
  6. 2024Efficient Neural Compression with Inference-time Decoding
  7. 2021SmartDeal: Re-Modeling Deep Network Weights for Efficient Inference and Training
  8. 2024Dynamic Memory Compression: Retrofitting LLMs for Accelerated Inference