mlsanityData to 7 Oct 2026

Compressing Neural Networks with the Hashing Trick

arXiv 1504.04788 ↗First posted 19 Apr 2015cs.LG cs.NEcited by 150

Not on the map. Why?

Abstract (arXiv)

As deep nets are increasingly used in applications suited for mobile devices, a fundamental dilemma becomes apparent: the trend in deep learning is to grow models to absorb ever-increasing data set sizes; however mobile devices are designed with very little memory and cannot store such large models. We present a novel network architecture, HashedNets, that exploits inherent redundancy in neural networks to achieve drastic reductions in model sizes. HashedNets uses a low-cost hash function to randomly group connection weights into hash buckets, and all connections within the same hash bucket share a single parameter value. These parameters are tuned to adjust to the HashedNets weight sharing architecture with standard backprop during training. Our hashing procedure introduces no additional memory overhead, and we demonstrate on several benchmark data sets that HashedNets shrink the storage requirements of neural networks substantially while mostly preserving generalization performance.

Key facts. Cited by 150 papers in the map (data through 7 Oct 2026). First posted on arXiv 19 Apr 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 36; 2022: 32; 2023: 36; 2024: 21; 2025: 13; 2026: 12.

150 papers.

Sort:

By area:

  1. 2022Not All Patches are What You Need: Expediting Vision Transformers via Token Reorganizationscited by 359
  2. 2021Pruning and Quantization for Deep Neural Network Acceleration: A Surveycited by 259
  3. 2021Hash Layers For Large Sparse Modelscited by 236
  4. 2024The Unreasonable Ineffectiveness of the Deeper Layerscited by 219
  5. 2023EAGLES: Efficient Accelerated 3D Gaussians with Lightweight EncodingScited by 191
  6. 2022Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communicationscited by 166
  7. 2021Distributed Learning in Wireless Networks: Recent Progress and Future Challengescited by 125
  8. 2021A Survey of Modern Deep Learning based Object Detection Modelscited by 97
  9. 2021Convolutional Neural Network Pruning with Structural Redundancy Reductioncited by 73
  10. 2023A Distributed Data-Parallel PyTorch Implementation of the Distributed Shampoo Optimizer for Training Neural Networks At-Scalecited by 69
  11. 2022A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classificationcited by 64
  12. 2021CDFI: Compression-Driven Network Design for Frame Interpolationcited by 55
38 more
  1. 2021Compressing Neural Networks: Towards Determining the Optimal Layer-wise Decompositioncited by 33
  2. 2021Multiplying Matrices Without Multiplyingcited by 30
  3. 2023NOLA: Compressing LoRA using Linear Combination of Random Basiscited by 28
  4. 2022Pruning Neural Networks via Coresets and Convex Geometry: Towards No Assumptionscited by 20
  5. 2023PowerQuant: Automorphism Search for Non-Uniform Quantizationcited by 20
  6. 2024Hash3D: Training-free Acceleration for 3D Generationcited by 19
  7. 2021Fast Walsh-Hadamard Transform and Smooth-Thresholding Based Binary Layers in Deep Neural Networkscited by 14
  8. 2021Connectivity Matters: Neural Network Pruning Through the Lens of Effective Sparsitycited by 14
  9. 2022DiSparse: Disentangled Sparsification for Multitask Model Compressioncited by 14
  10. 2023Unified Embedding: Battle-Tested Feature Representations for Web-Scale ML Systemscited by 14
  11. 2023Computation-efficient Deep Learning for Computer Vision: A Surveycited by 14
  12. 2024CoMERA: Computing- and Memory-Efficient Training via Rank-Adaptive Tensor Optimizationcited by 14
  13. 2024Sketch-GNN: Scalable Graph Neural Networks with Sublinear Training Complexitycited by 14
  14. 2022Block Walsh-Hadamard Transform Based Binary Layers in Deep Neural Networkscited by 13
  15. 2022Learning Compressed Embeddings for On-Device Inferencecited by 13
  16. 2022Heterogeneous Acceleration Pipeline for Recommendation System Trainingcited by 13
  17. 2022Dimensionality Reduced Training by Pruning and Freezing Parts of a Deep Neural Network, a Surveycited by 12
  18. 2022Interpretations Steered Network Pruning via Amortized Inferred Saliency Mapscited by 11
  19. 2023Massively Scaling Heteroscedastic Classifierscited by 11
  20. 2023OPORP: One Permutation + One Random Projectioncited by 11
  21. 2023Towards Fairer and More Efficient Federated Learning via Multidimensional Personalized Edge Modelscited by 10
  22. 2021Accelerating Neural ODEs Using Model Order Reductioncited by 9
  23. 2021Continual 3D Convolutional Neural Networks for Real-time Processing of Videoscited by 9
  24. 2023Globally Optimal Training of Neural Networks with Threshold Activation Functionscited by 9
  25. 2023Data-Free Quantization via Mixed-Precision Compensation without Fine-Tuningcited by 9
  26. 2024Jointly Training and Pruning CNNs via Learnable Agent Guidance and Alignmentcited by 9
  27. 2022Safety and Performance, Why not Both? Bi-Objective Optimized Model Compression toward AI Software Deploymentcited by 8
  28. 2022Learning Robust Dynamics through Variational Sparse Gatingcited by 8
  29. 2021Deep Model Compression based on the Training Historycited by 7
  30. 2021Semantically Constrained Memory Allocation (SCMA) for Embedding in Efficient Recommendation Systemscited by 7
  31. 2022LilNetX: Lightweight Networks with EXtreme Model Compression and Structured Sparsificationcited by 7
  32. 2023NUPES : Non-Uniform Post-Training Quantization via Power Exponent Searchcited by 7
  33. 2021COPS: Controlled Pruning Before Training Startscited by 6
  34. 2021Low-rank Tensor Decomposition for Compression of Convolutional Neural Networks Using Funnel Regularizationcited by 6
  35. 2022REx: Data-Free Residual Quantization Error Expansioncited by 6
  36. 2022A Theoretical View on Sparsely Activated Networkscited by 6
  37. 2022Sparsity-guided Network Design for Frame Interpolationcited by 6
  38. 2021Training Compact CNNs for Image Classification using Dynamic-coded Filter Fusioncited by 5

50 most cited of 150 All 150 (JSON)