# Neural Networks in Adversarial Setting and Ill-Conditioned Weight Space

arXiv: https://arxiv.org/abs/1801.00905 · HTML: https://mlsanity.com/paper/1801.00905/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jan 2018.

## Built on by (most cited first, 5 of 5)

- [SORSA: Singular Values and Orthonormal Regularized Singular Vectors Adaptation of Large Language Models](https://mlsanity.com/paper/2409.00055/) (2024)
- [Singular value decomposition based matrix surgery](https://mlsanity.com/paper/2302.11446/) (2023)
- [Lipschitz Constant Meets Condition Number: Learning Robust and Compact Deep Neural Networks](https://mlsanity.com/paper/2503.20454/) (2025)
- [Revealing Hidden Vulnerabilities in Autoencoders through Gradient Signal Restoration](https://mlsanity.com/paper/2505.03646/) (2025)
- [Algorithms and data structures for automatic precision estimation of neural networks](https://mlsanity.com/paper/2509.24607/) (2025)

## Abstract (arXiv)

Recently, Neural networks have seen a huge surge in its adoption due to their ability to provide high accuracy on various tasks. On the other hand, the existence of adversarial examples have raised suspicions regarding the generalization capabilities of neural networks. In this work, we focus on the weight matrix learnt by the neural networks and hypothesize that ill conditioned weight matrix is one of the contributing factors in neural network's susceptibility towards adversarial examples. For ensuring that the learnt weight matrix's condition number remains sufficiently low, we suggest using orthogonal regularizer. We show that this indeed helps in increasing the adversarial accuracy on MNIST and F-MNIST datasets.