mlsanityData to 14 Oct 2026

Learning Discrete Weights Using the Local Reparameterization Trick

arXiv 1710.07739 ↗First posted 21 Oct 2017cs.LG stat.MLcited by 29

Not on the map. Why?

Abstract (arXiv)

Recent breakthroughs in computer vision make use of large deep neural networks, utilizing the substantial speedup offered by GPUs. For applications running on limited hardware, however, high precision real-time processing can still be a challenge. One approach to solving this problem is training networks with binary or ternary weights, thus removing the need to calculate multiplications and significantly reducing memory size. In this work, we introduce LR-nets (Local reparameterization networks), a new method for training neural networks with discrete weights using stochastic parameters. We show how a simple modification to the local reparameterization trick, previously used to train Gaussian distributed weights, enables the training of discrete weights. Using the proposed training we test both binary and ternary models on MNIST, CIFAR-10 and ImageNet benchmarks and reach state-of-the-art results on most experiments.

Key facts. Cited by 29 papers in the map (data through 14 Oct 2026). First posted on arXiv 21 Oct 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 13; 2022: 7; 2023: 5; 2024: 2; 2025: 1; 2026: 1.

29 papers.

Sort:

By area:

  1. 2021Differentiable Model Compression via Pseudo Quantization Noisecited by 45
  2. 2021DKM: Differentiable K-Means Clustering Layer for Neural Network Compressioncited by 31
  3. 2021SiMaN: Sign-to-Magnitude Network Binarizationcited by 18
  4. 2021Cluster-Promoting Quantization with Bit-Drop for Minimizing Network Quantization Losscited by 12
  5. 2022Rethinking Bayesian Learning for Data Analysis: The Art of Prior and Inference in Sparsity-Aware Modelingcited by 12
  6. 2021Demystifying and Generalizing BinaryConnectcited by 11
  7. 2021Finding the Task-Optimal Low-Bit Sub-Distribution in Deep Neural Networkscited by 9
  8. 2021S3: Sign-Sparse-Shift Reparametrization for Effective Training of Low-bit Shift Networkscited by 7
  9. 2022Signing the Supermask: Keep, Hide, Invertcited by 7
  10. 2022Mixed-Precision Neural Networks: A Surveycited by 7
  11. 2021TyXe: Pyro-based Bayesian neural nets for Pytorchcited by 6
  12. 2021Federated Learning via Plurality Votecited by 6
17 more
  1. 2022Resource-efficient Deep Neural Networks for Automotive Radar Interference Mitigationcited by 6
  2. 2021Probabilistic Selective Encryption of Convolutional Neural Networks for Hierarchical Servicescited by 5
  3. 2022DenseShift: Towards Accurate and Efficient Low-Bit Power-of-Two Quantizationcited by 5
  4. 2021HEMP: High-order Entropy Minimization for neural network comPressioncited by 4
  5. 2021Bias-Variance Tradeoffs in Single-Sample Binary Gradient Estimatorscited by 4
  6. 2022Hyperspherical Quantization: Toward Smaller and More Accurate Modelscited by 4
  7. 2023Towards Optimal Compression: Joint Pruning and Quantizationcited by 4
  8. 2023Ternary Quantization: A Surveycited by 2
  9. 2023Efficient Learning of Discrete-Continuous Computation Graphscited by 2
  10. 2024One-Step Forward and Backtrack: Overcoming Zig-Zagging in Loss-Aware Quantization Trainingcited by 2
  11. 2023Learning Disentangled Discrete Representationscited by 1
  12. 2021Pruning Ternary Quantizationcited by 0
  13. 2022Nonlocal optimization of binary neural networkscited by 0
  14. 2023Learning Discrete Weights and Activations Using the Local Reparameterization Trickcited by 0
  15. 2024Efficient Model Compression for Bayesian Neural Networkscited by 0
  16. 2025SQS: Bayesian DNN Compression through Sparse Quantized Sub-distributionscited by 0
  17. 2026APQF: Agentic Profiling-Guided Structured Pruning and Mixed-Precision Quantization with Adaptive Fine-Tuningcited by 0