mlsanityData to 14 Oct 2026

ENEC: A Lossless AI Model Compression Method Enabling Fast Inference on Ascend NPUs

arXiv 2604.03298 ↗First posted 28 Mar 2026cs.AR cs.DC cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

The rapid scaling of Large Language Models presents significant challenges for their deployment and inference, particularly on resource-constrained specialized AI hardware accelerators such as Huawei's Ascend NPUs, where weight data transfer has become a critical performance bottleneck. While lossless compression can preserve model accuracy and reduce data volume, existing lossless compression algorithms exhibit extremely low throughput when ported to the Ascend NPU architecture. In this paper, we propose ENEC, a novel lossless compression method specifically customized for AI model weights and optimized for Ascend Neural Processing Units. ENEC adopts a block-based fixed-length encoding scheme and incorporates a series of NPU-specific optimizations: bit-width quantization with hierarchical halving bit-packing, vectorized branch-free integer transformation, and dependency-decoupled intra-segment scan for efficient prefix-sum computation. Experimental results demonstrate that ENEC outperforms existing state-of-the-art NPU compressors in both compression ratio and throughput. Compared to leading GPU solutions, ENEC achieves a 3.43X higher throughput than DietGPU and a 1.12X better compression ratio than nvCOMP. By reducing weight transmission overhead, ENEC significantly improves end-to-end inference performance, achieving up to a 6.3X speedup. On Ascend NPUs, ENEC is the first open-source lossless compression algorithm for model weights that achieves performance comparable to state-of-the-art GPU compressors, offering an effective solution for deploying large-scale AI models.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 36 of its 72 references that are in the map. First posted on arXiv 28 Mar 2026.

Builds on

36 of 72 references.

References · 36

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2025Qwen3 Technical Reportcited by 8,820
  5. 2023Mistral 7Bcited by 3,164
  6. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  7. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  8. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  9. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  10. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
  11. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  12. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 763
24 more
  1. 2024Phi-4 Technical Reportcited by 711
  2. 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermismcited by 642
  3. 2024OLMo: Accelerating the Science of Language Modelscited by 613
  4. 2020Unsupervised Cross-lingual Representation Learning for Speech Recognitioncited by 501
  5. 2024SliceGPT: Compress Large Language Models by Deleting Rows and Columnscited by 385
  6. 2024SVD-LLM: Truncation-aware Singular Value Decomposition for Large Language Model Compressioncited by 243
  7. 2019A Study of BFLOAT16 for Deep Learning Trainingcited by 232
  8. 2024A Survey on Efficient Inference for Large Language Modelscited by 205
  9. 2022FP8 Formats for Deep Learningcited by 196
  10. 2023ASVD: Activation-aware Singular Value Decomposition for Compressing Large Language Modelscited by 169
  11. 2024LLM Inference Unveiled: Survey and Roofline Model Insightscited by 152
  12. 2024MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUscited by 112
  13. 2024COAT: Compressing Optimizer states and Activation for Memory-Efficient FP8 Trainingcited by 33
  14. 202570% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float (DFloat11)cited by 24
  15. 2025Serving Large Language Models on Huawei CloudMatrix384cited by 24
  16. 2024SDP4Bit: Toward 4-bit Communication Quantization in Sharded Data Parallelism for LLM Trainingcited by 14
  17. 2024ZipNN: Lossless Compression for AI Modelscited by 14
  18. 2024To FP8 and Back Again: Quantifying Reduced Precision Effects on LLM Training Stabilitycited by 9
  19. 2025Huff-LLM: End-to-End Lossless Compression for Efficient LLM Inferencecited by 9
  20. 2025ElasticMM: Efficient Multimodal LLMs Serving with Elastic Multimodal Parallelismcited by 7
  21. 2023FCBench: Cross-Domain Benchmarking of Lossless Compression for Floating-Point Datacited by 5
  22. 2023Recoil: Parallel rANS Decoding with Decoder-Adaptive Scalabilitycited by 3
  23. 2025Lossless Compression of Neural Network Components: Weights, Checkpoints, and K/V Caches in Low-Precision Formatscited by 3
  24. 2023GPULZ: Optimizing LZSS Lossless Compression for Multi-byte Data on Modern GPUscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Lossless Tensor Compression as Program Synthesiscited by 1
  2. 2026HyperParallel-MoE: Multi-Core Interleaved Scheduling for Fast MoE Training on Ascend NPUscited by 0
  3. 2026Unified AI Gateway: A Framework for Joint Model Routing and KV Cache Managementcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025EntroLLM: Entropy Encoded Weight Compression for Efficient Large Language Model Inference on Edge Devices
  2. 2026ZipServ: Fast and Memory-Efficient LLM Inference with Hardware-Aware Lossless Compression
  3. 2024From Algorithm to Hardware: A Survey on Efficient and Safe Deployment of Deep Neural Networks
  4. 2025To Compress or Not? Pushing the Frontier of Lossless GenAI Model Weights Compression with Exponent Concentration
  5. 2026EntroPack: Fast and Accurate Entropy-Coded Weight Compression at Arbitrary Bitrates
  6. 2025Reimagining Memory Access for LLM Inference: Compression-Aware Memory Controller Design
  7. 2026LLM Compression with Jointly Optimizing Architectural and Quantization choices
  8. 2022Towards Hardware-Specific Automatic Compression of Neural Networks