Active Subspace of Neural Networks: Structural Analysis and Universal Attacks
Not on the map. Why?
Abstract (arXiv)
Active subspace is a model reduction method widely used in the uncertainty quantification community. In this paper, we propose analyzing the internal structure and vulnerability and deep neural networks using active subspace. Firstly, we employ the active subspace to measure the number of "active neurons" at each intermediate layer and reduce the number of neurons from several thousands to several dozens. This motivates us to change the network structure and to develop a new and more compact network, referred to as ASNet, that has significantly fewer model parameters. Secondly, we propose analyzing the vulnerability of a neural network using active subspace and finding an additive universal adversarial attack vector that can misclassify a dataset with a high probability. Our experiments on CIFAR-10 show that ASNet can achieve 23.98× parameter and 7.30× flops reduction. The universal active subspace attack vector can achieve around 20% higher attack ratio compared with the existing approach in all of our numerical experiments. The PyTorch codes for this paper are available online.
Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). First posted on arXiv 29 Oct 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 3; 2022: 5; 2023: 2; 2024: 0; 2025: 2.
12 papers.
- 2025CoLA: Compute-Efficient Pre-Training of LLMs via Low-Rank Activationcited by 28
- 2021Revisiting Knowledge Distillation: An Inheritance and Exploration Frameworkcited by 15
- 2022Few-Bit Backward: Quantized Gradients of Activation Functions for Memory Footprint Reductioncited by 14
- 2023Learning Active Subspaces for Effective and Scalable Uncertainty Quantification in Deep Neural Networkscited by 9
- 2025CR-Net: Scaling Parameter-Efficient Training with Cross-Layer Low-Rank Structurecited by 7
- 2022Federated Optimization of Smooth Loss Functionscited by 6
- 2023Quantization Aware Factorization for Deep Neural Network Compressioncited by 6
- 2022Gradient Descent for Low-Rank Functionscited by 3
- 2021A Dimensionality Reduction Approach for Convolutional Neural Networkscited by 2
- 2022Optimization Models and Interpretations for Three Types of Adversarial Perturbations against Support Vector Machinescited by 2
- 2021Tensor-based framework for training flexible neural networkscited by 0
- 2022A Proper Orthogonal Decomposition approach for parameters reduction of Single Shot Detector networkscited by 0
- 2025CR-Net: Scaling Parameter-Efficient Training with Cross-Layer Low-Rank Structure23 Sep 2025
- 2025CoLA: Compute-Efficient Pre-Training of LLMs via Low-Rank Activation16 Feb 2025
- 2023Learning Active Subspaces for Effective and Scalable Uncertainty Quantification in Deep Neural Networks6 Sep 2023
- 2023Quantization Aware Factorization for Deep Neural Network Compression8 Aug 2023
- 2022A Proper Orthogonal Decomposition approach for parameters reduction of Single Shot Detector networks27 Jul 2022
- 2022Gradient Descent for Low-Rank Functions16 Jun 2022
- 2022Optimization Models and Interpretations for Three Types of Adversarial Perturbations against Support Vector Machines7 Apr 2022
- 2022Few-Bit Backward: Quantized Gradients of Activation Functions for Memory Footprint Reduction1 Feb 2022
- 2022Federated Optimization of Smooth Loss Functions6 Jan 2022
- 2021A Dimensionality Reduction Approach for Convolutional Neural Networks18 Oct 2021
- 2021Revisiting Knowledge Distillation: An Inheritance and Exploration Framework1 Jul 2021
- 2021Tensor-based framework for training flexible neural networks25 Jun 2021