# Deep k-Means: Re-Training and Parameter Sharing with Harder Cluster Assignments for Compressing Deep Convolutions

arXiv: https://arxiv.org/abs/1806.09228 · HTML: https://mlsanity.com/paper/1806.09228/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jun 2018.

## Built on by (most cited first, 30 of 33)

- [HW-NAS-Bench:Hardware-Aware Neural Architecture Search Benchmark](https://mlsanity.com/paper/2103.10584/) (2021)
- [Empowering Edge Intelligence: A Comprehensive Survey on On-Device AI Models](https://mlsanity.com/paper/2503.06027/) (2025)
- [DKM: Differentiable K-Means Clustering Layer for Neural Network Compression](https://mlsanity.com/paper/2108.12659/) (2021)
- [Drawing Robust Scratch Tickets: Subnetworks with Inborn Robustness Are Found within Randomly Initialized Networks](https://mlsanity.com/paper/2110.14068/) (2021)
- [Model Compression in Practice: Lessons Learned from Practitioners Creating On-device Machine Learning Experiences](https://mlsanity.com/paper/2310.04621/) (2023)
- [CPT: Efficient Deep Neural Network Training via Cyclic Precision](https://mlsanity.com/paper/2101.09868/) (2021)
- [LOTUS: Evasive and Resilient Backdoor Attacks through Sub-Partitioning](https://mlsanity.com/paper/2403.17188/) (2024)
- [Auto-NBA: Efficient and Effective Search Over the Joint Space of Networks, Bitwidths, and Accelerators](https://mlsanity.com/paper/2106.06575/) (2021)
- [PDP: Parameter-free Differentiable Pruning is All You Need](https://mlsanity.com/paper/2305.11203/) (2023)
- [Efficient Neural Networks for Tiny Machine Learning: A Comprehensive Review](https://mlsanity.com/paper/2311.11883/) (2023)
- [Reducing Computational Complexity of Neural Networks in Optical Channel Equalization: From Concepts to Implementation](https://mlsanity.com/paper/2208.12866/) (2022)
- [Talaria: Interactively Optimizing Machine Learning Models for Efficient Inference](https://mlsanity.com/paper/2404.03085/) (2024)
- [eDKM: An Efficient and Accurate Train-time Weight Clustering for Large Language Models](https://mlsanity.com/paper/2309.00964/) (2023)
- [Bit-serial Weight Pools: Compression and Arbitrary Precision Execution of Neural Networks on Resource Constrained Processors](https://mlsanity.com/paper/2201.11651/) (2022)
- [FedCode: Communication-Efficient Federated Learning via Transferring Codebooks](https://mlsanity.com/paper/2311.09270/) (2023)
- [ViM-VQ: Efficient Post-Training Vector Quantization for Visual Mamba](https://mlsanity.com/paper/2503.09509/) (2025)
- [RT-RCG: Neural Network and Accelerator Search Towards Effective and Real-time ECG Reconstruction from Intracardiac Electrograms](https://mlsanity.com/paper/2111.02569/) (2021)
- [LDP: Learnable Dynamic Precision for Efficient Deep Neural Network Training and Inference](https://mlsanity.com/paper/2203.07713/) (2022)
- [Algorithmic Simplification of Neural Networks with Mosaic-of-Motifs](https://mlsanity.com/paper/2602.14896/) (2026)
- [AdaSpring: Context-adaptive and Runtime-evolutionary Deep Model Compression for Mobile Applications](https://mlsanity.com/paper/2101.11800/) (2021)
- [A New Clustering-Based Technique for the Acceleration of Deep Convolutional Networks](https://mlsanity.com/paper/2107.09095/) (2021)
- [Computational Complexity Evaluation of Neural Network Applications in Signal Processing](https://mlsanity.com/paper/2206.12191/) (2022)
- [Weight Fixing Networks](https://mlsanity.com/paper/2210.13554/) (2022)
- [Probabilistic Weight Fixing: Large-scale training of neural network weight uncertainties for quantization](https://mlsanity.com/paper/2309.13575/) (2023)
- [Separate, Dynamic and Differentiable (SMART) Pruner for Block/Output Channel Pruning on Computer Vision Tasks](https://mlsanity.com/paper/2403.19969/) (2024)
- [AdaScale: Dynamic Context-aware DNN Scaling via Automated Adaptation Loop on Mobile Devices](https://mlsanity.com/paper/2412.00724/) (2024)
- [LegoNet: Memory Footprint Reduction Through Block Weight Clustering](https://mlsanity.com/paper/2603.06606/) (2026)
- [SmartDeal: Re-Modeling Deep Network Weights for Efficient Inference and Training](https://mlsanity.com/paper/2101.01163/) (2021)
- [IDKM: Memory Efficient Neural Network Quantization via Implicit, Differentiable k-Means](https://mlsanity.com/paper/2312.07759/) (2023)
- [Accelerating spherical K-means clustering for large-scale sparse document data](https://mlsanity.com/paper/2411.11300/) (2024)

## Abstract (arXiv)

The current trend of pushing CNNs deeper with convolutions has created a pressing demand to achieve higher compression gains on CNNs where convolutions dominate the computation and parameter amount (e.g., GoogLeNet, ResNet and Wide ResNet). Further, the high energy consumption of convolutions limits its deployment on mobile devices. To this end, we proposed a simple yet effective scheme for compressing convolutions though applying k-means clustering on the weights, compression is achieved through weight-sharing, by only recording K cluster centers and weight assignment indexes. We then introduced a novel spectrally relaxed k-means regularization, which tends to make hard assignments of convolutional layer weights to K learned cluster centers during re-training. We additionally propose an improved set of metrics to estimate energy consumption of CNN hardware implementations, whose estimation results are verified to be consistent with previously proposed energy estimation tool extrapolated from actual hardware measurements. We finally evaluated Deep k-Means across several CNN models in terms of both compression ratio and energy consumption reduction, observing promising results without incurring accuracy loss. The code is available at https://github.com/Sandbox3aster/Deep-K-Means