mlsanityData to 7 Oct 2026

Sparsity-Probe: Analysis tool for Deep Learning Models

arXiv 2105.06849 ↗First posted 14 May 2021cs.LG math.FAcited by 4

Not on the map. Why?

Abstract (arXiv)

We propose a probe for the analysis of deep learning architectures that is based on machine learning and approximation theoretical principles. Given a deep learning architecture and a training set, during or after training, the Sparsity Probe allows to analyze the performance of intermediate layers by quantifying the geometrical features of representations of the training set. We show how the Sparsity Probe enables measuring the contribution of adding depth to a given architecture, to detect under-performing layers, etc., all this without any auxiliary test data set.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 26 of its 64 references that are in the map. First posted on arXiv 14 May 2021.

Builds on

26 of 64 references.

References · 26

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  7. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  8. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  9. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  10. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  11. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  12. 2020Supervised Contrastive Learningcited by 2,803
14 more
  1. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  2. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  3. 2018AutoAugment: Learning Augmentation Policies from Datacited by 728
  4. 2017Deep Neural Networks as Gaussian Processescited by 617
  5. 2016Error bounds for approximations with deep ReLU networkscited by 567
  6. 2019Fantastic Generalization Measures and Where to Find Themcited by 507
  7. 2017Opening the Black Box of Deep Neural Networks via Informationcited by 494
  8. 2018Optimal approximation of continuous functions by very deep ReLU networkscited by 165
  9. 2020Deep Network Approximation for Smooth Functionscited by 163
  10. 2016Improving the Robustness of Deep Neural Networks via Stability Trainingcited by 156
  11. 2019Deep Network Approximation Characterized by Number of Neuronscited by 120
  12. 2016Convolutional Neural Networks Analyzed via Convolutional Sparse Codingcited by 51
  13. 2020Scaling Down Deep Learning with MNIST-1Dcited by 23
  14. 2017Multi-Layer Convolutional Sparse Modeling: Pursuit and Dictionary Learningcited by 14

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 1; 2024: 0; 2025: 1.

4 papers.

  1. 2023Reverse Engineering Self-Supervised Learningcited by 33
  2. 2022Nearest Class-Center Simplification through Intermediate Layerscited by 20
  3. 2022On the Implicit Bias Towards Minimal Depth of Deep Neural Networkscited by 12
  4. 2025Transforming Datasets to Requested Complexity with Projection-based Many-Objective Genetic Algorithmcited by 1