mlsanityData to 14 Oct 2026

Soft Quantization: Model Compression Via Weight Coupling

arXiv 2601.21219 ↗First posted 29 Jan 2026cs.LG cond-mat.dis-nncited by 0

Not on the map. Why?

Abstract (arXiv)

We show that introducing short-range attractive couplings between the weights of a neural network during training provides a novel avenue for model quantization. These couplings rapidly induce the discretization of a model's weight distribution, and they do so in a mixed-precision manner despite only relying on two additional hyperparameters. We demonstrate that, within an appropriate range of hyperparameters, our "soft quantization'' scheme outperforms histogram-equalized post-training quantization on ResNet-20/CIFAR-10. Soft quantization provides both a new pipeline for the flexible compression of machine learning models and a new tool for investigating the trade-off between compression and generalization in high-dimensional loss landscapes.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 12 of its 18 references that are in the map. First posted on arXiv 29 Jan 2026.

Builds on

12 of 18 references.

References · 12

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  3. 2018Do Better ImageNet Models Transfer Better?cited by 565
  4. 2021A White Paper on Neural Network Quantizationcited by 472
  5. 20218-bit Optimizers via Block-wise Quantizationcited by 403
  6. 2019Understanding Straight-Through Estimator in Training Activation Quantized Neural Netscited by 231
  7. 2023Cross-Entropy Loss Functions: Theoretical Analysis and Applicationscited by 210
  8. 2016What makes ImageNet good for transfer learning?cited by 192
  9. 2017Soft Weight-Sharing for Neural Network Compressioncited by 99
  10. 2024A Comprehensive Evaluation of Quantization Strategies for Large Language Modelscited by 48
  11. 2018BinaryRelax: A Relaxation Approach For Training Deep Neural Networks With Quantized Weightscited by 24
  12. 2025Histogram-Equalized Quantization for logic-gated Residual Neural Networkscited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2021Soft then Hard: Rethinking the Quantization in Neural Image Compression
  2. 2026SoftWater: Class-Aware Rate Allocation for Softmax Quantization
  3. 2026Quantization-Aware Regularizers for Deep Neural Networks Compression
  4. 2025CoDeQ: End-to-End Joint Model Compression with Dead-Zone Quantizer for High-Sparsity and Low-Precision Networks
  5. 2024Retraining-free Model Quantization via One-Shot Weight-Coupling Learning
  6. 2023RepQ: Generalizing Quantization-Aware Training for Re-Parametrized Architectures
  7. 2021Compact representations of convolutional neural networks via weight pruning and quantization
  8. 2024SLiM: One-shot Quantization and Sparsity with Low-rank Approximation for LLM Weight Compression