mlsanityData to 14 Oct 2026

Depthwise Separable Convolutions Allow for Fast and Memory-Efficient Spectral Normalization

arXiv 2102.06496 ↗First posted 12 Feb 2021cs.LG cs.CV stat.MLcited by 1

Not on the map. Why?

Abstract (arXiv)

An increasing number of models require the control of the spectral norm of convolutional layers of a neural network. While there is an abundance of methods for estimating and enforcing upper bounds on those during training, they are typically costly in either memory or time. In this work, we introduce a very simple method for spectral normalization of depthwise separable convolutions, which introduces negligible computational and memory overhead. We demonstrate the effectiveness of our method on image classification tasks using standard architectures like MobileNetV2.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 26 of its 35 references that are in the map. First posted on arXiv 12 Feb 2021.

Builds on

26 of 35 references.

References · 26

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  4. 2013Intriguing properties of neural networkscited by 5,155
  5. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  6. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  7. 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,332
  8. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,636
  9. 2014NICE: Non-linear Independent Components Estimationcited by 1,058
  10. 2018Lipschitz regularity of deep neural networks: analysis and efficient estimationcited by 415
  11. 2020Meta Pseudo Labelscited by 374
  12. 2018Invertible Residual Networkscited by 365
14 more
  1. 2018Regularisation of Neural Networks by Enforcing Lipschitz Continuitycited by 314
  2. 2017Parseval Networks: Improving Robustness to Adversarial Examplescited by 293
  3. 2019Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networkscited by 293
  4. 2017The Reversible Residual Network: Backpropagation Without Storing Activationscited by 217
  5. 2018Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networkscited by 195
  6. 2017Spectral Norm Regularization for Improving the Generalizability of Deep Learningcited by 182
  7. 2018Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networkscited by 176
  8. 2018The Singular Values of Convolutional Layerscited by 95
  9. 2019Preventing Gradient Attenuation in Lipschitz Constrained Convolutional Networkscited by 87
  10. 2020Lipschitz constant estimation of Neural Networks via sparse polynomial optimizationcited by 87
  11. 2020Understanding and Mitigating Exploding Inverses in Invertible Neural Networkscited by 71
  12. 2018Generalizable Adversarial Training via Spectral Normalizationcited by 68
  13. 2018L2-Nonexpansive Neural Networkscited by 21
  14. 2018Lipschitz regularized Deep Neural Networks generalize and are adversarially robustcited by 19

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Smooth Model Compression without Fine-Tuningcited by 0