mlsanityData to 7 Oct 2026

Quantitative Performance Assessment of CNN Units via Topological Entropy Calculation

arXiv 2103.09716 ↗First posted 17 Mar 2021cs.CV cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Identifying the status of individual network units is critical for understanding the mechanism of convolutional neural networks (CNNs). However, it is still challenging to reliably give a general indication of unit status, especially for units in different network models. To this end, we propose a novel method for quantitatively clarifying the status of single unit in CNN using algebraic topological tools. Unit status is indicated via the calculation of a defined topological-based entropy, called feature entropy, which measures the degree of chaos of the global spatial pattern hidden in the unit for a category. In this way, feature entropy could provide an accurate indication of status for units in different networks with diverse situations like weight-rescaling operation. Further, we show that feature entropy decreases as the layer goes deeper and shares almost simultaneous trend with loss during training. We show that by investigating the feature entropy of units on only training data, it could give discrimination between networks with different generalization ability from the view of the effectiveness of feature representations.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 35 of its 44 references that are in the map. First posted on arXiv 17 Mar 2021.

Builds on

35 of 44 references.

References · 35

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2017Mask R-CNNcited by 7,259
  4. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  5. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  6. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  7. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  8. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  9. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  10. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  11. 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 703
  12. 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 697
23 more
  1. 2017ThiNet: A Filter Level Pruning Method for Deep Neural Network Compressioncited by 438
  2. 2014Understanding Deep Image Representations by Inverting Themcited by 399
  3. 2018Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Accelerationcited by 365
  4. 2019High Frequency Component Helps Explain the Generalization of Convolutional Neural Networkscited by 310
  5. 2020Understanding the Role of Individual Units in a Deep Neural Networkcited by 254
  6. 2015Persistence Images: A Stable Vector Representation of Persistent Homologycited by 235
  7. 2017NISP: Pruning Networks using Neuron Importance Score Propagationcited by 228
  8. 2016Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architecturescited by 220
  9. 2014Object Detectors Emerge in Deep Scene CNNscited by 178
  10. 2017Interpretable Convolutional Neural Networkscited by 161
  11. 2015A roadmap for the computation of persistent homologycited by 126
  12. 2015Path-SGD: Path-Normalized Optimization in Deep Neural Networkscited by 117
  13. 2017Deep Learning with Topological Signaturescited by 113
  14. 2020Topology of deep neural networkscited by 111
  15. 2017Interpreting Deep Visual Representations via Network Dissectioncited by 103
  16. 2018On the importance of single directions for generalizationcited by 99
  17. 2018Neural Persistence: A Complexity Measure for Deep Neural Networks Using Algebraic Topologycited by 73
  18. 2018Exploiting Kernel Sparsity and Entropy for Interpretable CNN Compressioncited by 40
  19. 2018On Characterizing the Capacity of Neural Networks using Algebraic Topologycited by 37
  20. 2018Revisiting the Importance of Individual Units in CNNs via Ablationcited by 32
  21. 2019Persistence Homology of Networks: Methods and Applicationscited by 30
  22. 2018Exposition and Interpretation of the Topology of Neural Networkscited by 20
  23. 2020Can neural networks learn persistent homology features?cited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 1; 2023: 0; 2024: 1; 2025: 2; 2026: 1.

6 papers.

  1. 2024Feature Map Convergence Evaluation for Functional Modulecited by 6
  2. 2025Applications of Entropy in Data Analysis and Machine Learning: A Reviewcited by 3
  3. 2021Estimating the Generalization in Deep Neural Networks via Sparsitycited by 2
  4. 2022Detecting danger in gridworlds using Gromov's Link Conditioncited by 1
  5. 2025FMCE-Net++: Feature Map Convergence Evaluation and Trainingcited by 0
  6. 2026Symmetric Divergence and Normalized Similarity: A Unified Topological Framework for Representation Analysiscited by 0