mlsanityData to 7 Oct 2026

Cauchy-Schwarz Regularizers

arXiv 2503.01639 ↗First posted 3 Mar 2025math.OC cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We introduce a novel class of regularization functions, called Cauchy-Schwarz (CS) regularizers, which can be designed to induce a wide range of properties in solution vectors of optimization problems. To demonstrate the versatility of CS regularizers, we derive regularization functions that promote discrete-valued vectors, eigenvectors of a given matrix, and orthogonal matrices. The resulting CS regularizers are simple, differentiable, and can be free of spurious stationary points, making them suitable for gradient-based solvers and large-scale optimization problems. In addition, CS regularizers automatically adapt to the appropriate scale, which is, for example, beneficial when discretizing the weights of neural networks. To demonstrate the efficacy of CS regularizers, we provide results for solving underdetermined systems of linear equations and weight quantization in neural networks. Furthermore, we discuss specializations, variations, and generalizations, which lead to an even broader class of new and possibly more powerful regularizers.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 31 of its 59 references that are in the map. First posted on arXiv 3 Mar 2025.

Builds on

31 of 59 references.

References · 31

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  4. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 829
  5. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 557
  6. 2016Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activationscited by 413
  7. 2018LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networkscited by 272
  8. 2019Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networkscited by 234
  9. 2016Ternary Weight Networkscited by 195
  10. 2019Forward and Backward Information Retention for Accurate Binary Neural Networkscited by 169
  11. 2018Learning to Quantize Deep Networks by Optimizing Quantization Intervals with Task Losscited by 157
  12. 2019Quantization Networkscited by 151
19 more
  1. 2017Deep Learning with Low Precision by Half-wave Gaussian Quantizationcited by 115
  2. 2017Extremely Low Bit Neural Network: Squeeze the Last Bit Out with ADMMcited by 60
  3. 2016Loss-aware Binarization of Deep Networkscited by 56
  4. 2018ProxQuant: Quantized Neural Networks via Proximal Operatorscited by 56
  5. 2021BSQ: Exploring Bit-Level Sparsity for Mixed-Precision Neural Network Quantizationcited by 37
  6. 2018Projection Convolutional Neural Networks for 1-bit CNNs via Discrete Back Propagationcited by 29
  7. 2018From Hashing to CNNs: Training BinaryWeight Networks via Hashingcited by 23
  8. 2017Learning Accurate Low-Bit Deep Neural Networks with Stochastic Quantizationcited by 21
  9. 2021Distance-aware Quantizationcited by 19
  10. 2022Lipschitz Continuity Retained Binary Neural Networkcited by 14
  11. 2018Regularized Binary Network Trainingcited by 13
  12. 2021Cauchy-Schwarz Regularized Autoencodercited by 13
  13. 2023Resilient Binary Neural Networkcited by 13
  14. 2018On Periodic Functions as Regularizers for Quantization of Neural Networkscited by 8
  15. 2019Adaptive Binary-Ternary Quantizationcited by 7
  16. 2022Improved Imaging by Invex Regularizers with Global Optima Guaranteescited by 6
  17. 2020WaveQ: Gradient-Based Deep Quantization of Neural Networks through Sinusoidal Adaptive Regularizationcited by 4
  18. 20171-bit Massive MU-MIMO Precoding in VLSIcited by 3
  19. 2023Invex Programs: First Order Algorithms and Their Convergencecited by 1

Built on by

Not cited yet.