mlsanityData to 7 Oct 2026

Understanding Convolutional Neural Networks with A Mathematical Model

arXiv 1609.04112 ↗First posted 14 Sep 2016cs.CVcited by 33

Not on the map. Why?

Abstract (arXiv)

This work attempts to address two fundamental questions about the structure of the convolutional neural networks (CNN): 1) why a non-linear activation function is essential at the filter output of every convolutional layer? 2) what is the advantage of the two-layer cascade system over the one-layer system? A mathematical model called the "REctified-COrrelations on a Sphere" (RECOS) is proposed to answer these two questions. After the CNN training process, the converged filter weights define a set of anchor vectors in the RECOS model. Anchor vectors represent the frequently occurring patterns (or the spectral components). The necessity of rectification is explained using the RECOS model. Then, the behavior of a two-layer RECOS system is analyzed and compared with its one-layer counterpart. The LeNet-5 and the MNIST dataset are used to illustrate discussion points. Finally, the RECOS model is generalized to a multi-layer system with the AlexNet as an example. Keywords: Convolutional Neural Network (CNN), Nonlinear Activation, RECOS Model, Rectified Linear Unit (ReLU), MNIST Dataset.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Sep 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 11; 2022: 9; 2023: 5; 2024: 4; 2025: 2; 2026: 2.

33 papers.

Sort:

  1. 2021DefakeHop: A Light-Weight High-Performance Deepfake Detectorcited by 31
  2. 2022Green Learning: Introduction, Examples and Outlookcited by 30
  3. 2021VoxelHop: Successive Subspace Learning for ALS Disease Classification Using Structural MRIcited by 18
  4. 2021AnomalyHop: An SSL-based Image Anomaly Localization Methodcited by 12
  5. 2021Successive Subspace Learning: An Overviewcited by 11
  6. 2021E-PixelHop: An Enhanced PixelHop Method for Object Classificationcited by 9
  7. 2021TGHop: An Explainable, Efficient and Lightweight Method for Texture Generationcited by 8
  8. 2021Segmentation of Cardiac Structures via Successive Subspace Learning with Saab Transform from Cine MRIcited by 7
  9. 2021GSIP: Green Semantic Segmentation of Large-Scale Indoor Point Cloudscited by 7
  10. 2021Convolutional Neural Networks Demystified: A Matched Filtering Perspective Based Tutorialcited by 6
  11. 2022DefakeHop++: An Enhanced Lightweight Deepfake Detectorcited by 5
  12. 2022PAGER: Progressive Attribute-Guided Extendable Robust Image Generationcited by 5
21 more
  1. 2021Bridging Gap between Image Pixels and Semantics via Supervision: A Surveycited by 4
  2. 2022A data filling methodology for time series based on CNN and (Bi)LSTM neural networkscited by 4
  3. 2022Subspace Learning Machine (SLM): Methodology and Performancecited by 4
  4. 2022GENHOP: An Image Generation Method Based on Successive Subspace Learningcited by 4
  5. 2021Understanding the Basis of Graph Convolutional Neural Networks via an Intuitive Matched Filtering Approachcited by 3
  6. 2022Design of Supervision-Scalable Learning Systems: Methodology and Performance Benchmarkingcited by 3
  7. 2023Enhancing Edge Intelligence with Highly Discriminant LNT Featurescited by 3
  8. 2024Demystifying the Effect of Receptive Field Size in U-Net Models for Medical Image Segmentationcited by 3
  9. 2025Trends and Challenges in Authorship Analysis: A Review of ML, DL, and LLM Approachescited by 2
  10. 2022Statistical Attention Localization (SAL): Methodology and Application to Object Classificationcited by 1
  11. 2022VS-CAM: Vertex Semantic Class Activation Mapping to Interpret Vision Graph Neural Networkcited by 1
  12. 2023Green Steganalyzer: A Green Learning Approach to Image Steganalysiscited by 1
  13. 2026HyperBank: A Differentiable Bank of Classical Priors for Few-Shot Spheroid Microscopy Segmentationcited by 1
  14. 2023Successive Subspace Learning for Cardiac Disease Classification with Two-phase Deformation Fields from Cine MRIcited by 0
  15. 2023LSR: A Light-Weight Super-Resolution Methodcited by 0
  16. 2023A Tiny Machine Learning Model for Point Cloud Object Classificationcited by 0
  17. 2024GWPT: A Green Word-Embedding-based POS Taggercited by 0
  18. 2024GAMA-IR: Global Additive Multidimensional Averaging for Fast Image Restorationcited by 0
  19. 2024On filter design in deep convolutional neural networkcited by 0
  20. 2025Lightweight Multi-Scale Feature Extraction with Fully Connected LMF Layer for Salient Object Detectioncited by 0
  21. 2026Interpretable and backpropagation-free Green Learning for efficient multi-task echocardiographic segmentation and classificationcited by 0