# HALOC: Hardware-Aware Automatic Low-Rank Compression for Compact Neural Networks

arXiv: https://arxiv.org/abs/2301.09422 · HTML: https://mlsanity.com/paper/2301.09422/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 55 references that are in the map.
- First posted on arXiv 20 Jan 2023.

## Builds on (most cited first, 30 of 31)

- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks](https://mlsanity.com/paper/1603.05279/) (2016)
- [Channel Pruning for Accelerating Very Deep Neural Networks](https://mlsanity.com/paper/1707.06168/) (2017)
- [ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware](https://mlsanity.com/paper/1812.00332/) (2018)
- [FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search](https://mlsanity.com/paper/1812.03443/) (2018)
- [Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation](https://mlsanity.com/paper/1404.0736/) (2014)
- [HRank: Filter Pruning using High-Rank Feature Map](https://mlsanity.com/paper/2002.10179/) (2020)
- [LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks](https://mlsanity.com/paper/1807.10029/) (2018)
- [Tensorizing Neural Networks](https://mlsanity.com/paper/1509.06569/) (2015)
- [Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition](https://mlsanity.com/paper/1412.6553/) (2014)
- [Discrimination-aware Channel Pruning for Deep Neural Networks](https://mlsanity.com/paper/1810.11809/) (2018)
- [Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applications](https://mlsanity.com/paper/1511.06530/) (2015)
- [SCOP: Scientific Control for Reliable Neural Network Pruning](https://mlsanity.com/paper/2010.10732/) (2020)
- [CHIP: CHannel Independence-based Pruning for Compact Neural Networks](https://mlsanity.com/paper/2110.13981/) (2021)
- [Stable Low-rank Tensor Decomposition for Compression of Convolutional Neural Network](https://mlsanity.com/paper/2008.05441/) (2020)
- [Towards Efficient Tensor Decomposition-Based DNN Model Compression with Optimization Framework](https://mlsanity.com/paper/2107.12422/) (2021)
- [Understanding Architectures Learnt by Cell-based Neural Architecture Search](https://mlsanity.com/paper/1909.09569/) (2019)
- [CHEX: CHannel EXploration for CNN Model Compression](https://mlsanity.com/paper/2203.15794/) (2022)
- [Compressing Recurrent Neural Networks with Tensor Ring for Action Recognition](https://mlsanity.com/paper/1811.07503/) (2018)
- [Compressing Neural Networks: Towards Determining the Optimal Layer-wise Decomposition](https://mlsanity.com/paper/2107.11442/) (2021)
- [CP-decomposition with Tensor Power Method for Convolutional Neural Networks Compression](https://mlsanity.com/paper/1701.07148/) (2017)
- [DECORE: Deep Compression with Reinforcement Learning](https://mlsanity.com/paper/2106.06091/) (2021)
- [Heuristic Rank Selection with Progressively Searching Tensor Ring Network](https://mlsanity.com/paper/2009.10580/) (2020)
- [Coordinating Filters for Faster Deep Neural Networks](https://mlsanity.com/paper/1703.09746/) (2017)
- [Deeper Insights into Weight Sharing in Neural Architecture Search](https://mlsanity.com/paper/2001.01431/) (2020)
- [RTMobile: Beyond Real-Time Mobile Acceleration of RNNs for Speech Recognition](https://mlsanity.com/paper/2002.11474/) (2020)
- [Compressing Recurrent Neural Networks Using Hierarchical Tucker Tensor Decomposition](https://mlsanity.com/paper/2005.04366/) (2020)
- [Towards Extremely Compact RNNs for Video Recognition with Fully Decomposed Hierarchical Tucker Structure](https://mlsanity.com/paper/2104.05758/) (2021)
- [TDC: Towards Extremely Efficient CNNs on GPUs via Hardware-Aware Tucker Decomposition](https://mlsanity.com/paper/2211.03715/) (2022)

## Built on by (most cited first, 20 of 20)

- [TopV: Compatible Token Pruning with Inference Time Optimization for Fast and Low-Memory Multimodal Vision Language Model](https://mlsanity.com/paper/2503.18278/) (2025)
- [A Survey of Token Compression for Efficient Multimodal Large Language Models](https://mlsanity.com/paper/2507.20198/) (2025)
- [A Review of Pseudo-Labeling for Computer Vision](https://mlsanity.com/paper/2408.07221/) (2024)
- [COMCAT: Towards Efficient Compression and Customization of Attention-Based Vision Models](https://mlsanity.com/paper/2305.17235/) (2023)
- [COAP: Memory-Efficient Training with Correlation-Aware Gradient Projection](https://mlsanity.com/paper/2412.00071/) (2024)
- [Edge AI: Evaluation of Model Compression Techniques for Convolutional Neural Networks](https://mlsanity.com/paper/2409.02134/) (2024)
- [Vision Technologies with Applications in Traffic Surveillance Systems: A Holistic Survey](https://mlsanity.com/paper/2412.00348/) (2024)
- [FLORA: Fine-grained Low-Rank Architecture Search for Vision Transformer](https://mlsanity.com/paper/2311.03912/) (2023)
- [Structure-Preserving Network Compression Via Low-Rank Induced Training Through Linear Layers Composition](https://mlsanity.com/paper/2405.03089/) (2024)
- [An Adaptive Tensor-Train Decomposition Approach for Efficient Deep Neural Network Compression](https://mlsanity.com/paper/2408.01534/) (2024)
- [Hardware-Aware DNN Compression for Homogeneous Edge Devices](https://mlsanity.com/paper/2501.15240/) (2025)
- [LinDeps: A Fine-tuning Free Post-Pruning Method to Remove Layer-Wise Linear Dependencies with Guaranteed Performance Preservation](https://mlsanity.com/paper/2507.21573/) (2025)
- [EcoSpa: Efficient Transformer Training with Coupled Sparsity](https://mlsanity.com/paper/2511.11641/) (2025)
- [Convolutional Neural Network Compression Based on Low-Rank Decomposition](https://mlsanity.com/paper/2408.16289/) (2024)
- [Unified Framework for Pre-trained Neural Network Compression via Decomposition and Optimized Rank Selection](https://mlsanity.com/paper/2409.03555/) (2024)
- [AdaRing: Towards Ultra-Light Vision-Language Adaptation via Cross-Layer Tensor Ring Decomposition](https://mlsanity.com/paper/2508.11870/) (2025)
- [Efficient CNN Compression via Multi-method Low Rank Factorization and Feature Map Similarity](https://mlsanity.com/paper/2510.00062/) (2025)
- [InstructMoLE: Instruction-Guided Mixture of Low-rank Experts for Multi-Conditional Image Generation](https://mlsanity.com/paper/2512.21788/) (2025)
- [Why Smaller Is Slower? Dimensional Misalignment in Compressed LLMs](https://mlsanity.com/paper/2604.09595/) (2026)
- [Large Models for Small Devices: Recent Advances and Empirical Analysis of Edge AI Deployment](https://mlsanity.com/paper/2608.15693/) (2026)

## Abstract (arXiv)

Low-rank compression is an important model compression strategy for obtaining compact neural network models. In general, because the rank values directly determine the model complexity and model accuracy, proper selection of layer-wise rank is very critical and desired. To date, though many low-rank compression approaches, either selecting the ranks in a manual or automatic way, have been proposed, they suffer from costly manual trials or unsatisfied compression performance. In addition, all of the existing works are not designed in a hardware-aware way, limiting the practical performance of the compressed models on real-world hardware platforms. To address these challenges, in this paper we propose HALOC, a hardware-aware automatic low-rank compression framework. By interpreting automatic rank selection from an architecture search perspective, we develop an end-to-end solution to determine the suitable layer-wise ranks in a differentiable and hardware-aware way. We further propose design principles and mitigation strategy to efficiently explore the rank space and reduce the potential interference problem. Experimental results on different datasets and hardware platforms demonstrate the effectiveness of our proposed approach. On CIFAR-10 dataset, HALOC enables 0.07% and 0.38% accuracy increase over the uncompressed ResNet-20 and VGG-16 models with 72.20% and 86.44% fewer FLOPs, respectively. On ImageNet dataset, HALOC achieves 0.9% higher top-1 accuracy than the original ResNet-18 model with 66.16% fewer FLOPs. HALOC also shows 0.66% higher top-1 accuracy increase than the state-of-the-art automatic low-rank compression solution with fewer computational and memory costs. In addition, HALOC demonstrates the practical speedups on different hardware platforms, verified by the measurement results on desktop GPU, embedded GPU and ASIC accelerator.