mlsanityData to 14 Oct 2026

Controlling the Complexity and Lipschitz Constant improves polynomial nets

arXiv 2202.05068 ↗First posted 10 Feb 2022cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

While the class of Polynomial Nets demonstrates comparable performance to neural networks (NN), it currently has neither theoretical generalization characterization nor robustness guarantees. To this end, we derive new complexity bounds for the set of Coupled CP-Decomposition (CCP) and Nested Coupled CP-decomposition (NCP) models of Polynomial Nets in terms of the ℓ_∞-operator-norm and the ℓ2-operator norm. In addition, we derive bounds on the Lipschitz constant for both models to establish a theoretical certificate for their robustness. The theoretical results enable us to propose a principled regularization scheme that we also evaluate experimentally in six datasets and show that it improves the accuracy as well as the robustness of the models to adversarial perturbations. We showcase how this regularization can be combined with adversarial training, resulting in further improvements.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 27 of its 56 references that are in the map. First posted on arXiv 10 Feb 2022.

Builds on

27 of 56 references.

References · 27

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  2. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  3. 2013Intriguing properties of neural networkscited by 5,155
  4. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  5. 2017Non-local Neural Networkscited by 2,293
  6. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,636
  7. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
  8. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
  9. 2017Spectrally-normalized margin bounds for neural networkscited by 686
  10. 2018Deep Learning for Classical Japanese Literaturecited by 422
  11. 2018Lipschitz regularity of deep neural networks: analysis and efficient estimationcited by 415
  12. 2017Defense against Adversarial Attacks Using High-Level Representation Guided Denoisercited by 329
15 more
  1. 2017A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networkscited by 314
  2. 2018Feature Denoising for Improving Adversarial Robustnesscited by 314
  3. 2015Norm-Based Capacity Control in Neural Networkscited by 299
  4. 2017Parseval Networks: Improving Robustness to Adversarial Examplescited by 293
  5. 2019Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networkscited by 293
  6. 2017Size-Independent Sample Complexity of Neural Networkscited by 290
  7. 2017Neural Audio Synthesis of Musical Notes with WaveNet Autoencoderscited by 225
  8. 2017EMNIST: an extension of MNIST to handwritten letterscited by 213
  9. 2015The MegaFace Benchmark: 1 Million Faces for Recognition at Scalecited by 200
  10. 2018Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networkscited by 195
  11. 2017Spectral Norm Regularization for Improving the Generalizability of Deep Learningcited by 182
  12. 2019Robust Learning with Jacobian Regularizationcited by 129
  13. 2017Implicit Regularization in Deep Learningcited by 90
  14. 2020Lipschitz constant estimation of Neural Networks via sparse polynomial optimizationcited by 87
  15. 2020Convolutional Tensor-Train LSTM for Spatio-temporal Learningcited by 55

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 1; 2024: 0; 2025: 1.

5 papers.

  1. 2023Regularization of polynomial networks for image recognitioncited by 13
  2. 2025Hadamard product in deep learning: Introduction, Advances and Challengescited by 9
  3. 2022Extrapolation and Spectral Bias of Neural Nets with Hadamard Product: a Polynomial Net Studycited by 7
  4. 2022Sound and Complete Verification of Polynomial Networkscited by 4
  5. 2022On the Study of Sample Complexity for Polynomial Neural Networkscited by 1