# Design of Supervision-Scalable Learning Systems: Methodology and Performance Benchmarking

arXiv: https://arxiv.org/abs/2206.09061 · HTML: https://mlsanity.com/paper/2206.09061/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 36 references that are in the map.
- First posted on arXiv 18 Jun 2022.

## Builds on (most cited first, 19 of 19)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Meta-Transfer Learning for Few-Shot Learning](https://mlsanity.com/paper/1812.02391/) (2018)
- [Meta-Baseline: Exploring Simple Meta-Learning for Few-Shot Learning](https://mlsanity.com/paper/2003.04390/) (2020)
- [Scalable Active Learning for Object Detection](https://mlsanity.com/paper/2004.04699/) (2020)
- [Interpretable Convolutional Neural Networks via Feedforward Design](https://mlsanity.com/paper/1810.02786/) (2018)
- [PointHop: An Explainable Machine Learning Method for Point Cloud Classification](https://mlsanity.com/paper/1907.12766/) (2019)
- [PixelHop++: A Small Successive-Subspace-Learning-Based (SSL-based) Model for Image Classification](https://mlsanity.com/paper/2002.03141/) (2020)
- [Understanding Convolutional Neural Networks with A Mathematical Model](https://mlsanity.com/paper/1609.04112/) (2016)
- [DefakeHop: A Light-Weight High-Performance Deepfake Detector](https://mlsanity.com/paper/2103.06929/) (2021)
- [R-PointHop: A Green, Accurate, and Unsupervised Point Cloud Registration Method](https://mlsanity.com/paper/2103.08129/) (2021)
- [On Supervised Feature Selection from High Dimensional Feature Spaces](https://mlsanity.com/paper/2203.11924/) (2022)
- [On Data-Driven Saak Transform](https://mlsanity.com/paper/1710.04176/) (2017)
- [PointHop++: A Lightweight Learning Model on Point Sets for 3D Classification](https://mlsanity.com/paper/2002.03281/) (2020)
- [Unsupervised Point Cloud Registration via Salient Points Analysis (SPA)](https://mlsanity.com/paper/2009.01293/) (2020)
- [Unsupervised Feedforward Feature (UFF) Learning for Point Cloud Classification and Segmentation](https://mlsanity.com/paper/2009.01280/) (2020)
- [FaceHop: A Light-Weight Low-Resolution Face Gender Classification Method](https://mlsanity.com/paper/2007.09510/) (2020)
- [AnomalyHop: An SSL-based Image Anomaly Localization Method](https://mlsanity.com/paper/2105.03797/) (2021)
- [Successive Subspace Learning: An Overview](https://mlsanity.com/paper/2103.00121/) (2021)
- [E-PixelHop: An Enhanced PixelHop Method for Object Classification](https://mlsanity.com/paper/2107.02966/) (2021)

## Built on by (most cited first, 3 of 3)

- [Green Learning: Introduction, Examples and Outlook](https://mlsanity.com/paper/2210.00965/) (2022)
- [Enhancing Edge Intelligence with Highly Discriminant LNT Features](https://mlsanity.com/paper/2312.14968/) (2023)
- [Acceleration of Subspace Learning Machine via Particle Swarm Optimization and Parallel Processing](https://mlsanity.com/paper/2208.07023/) (2022)

## Abstract (arXiv)

The design of robust learning systems that offer stable performance under a wide range of supervision degrees is investigated in this work. We choose the image classification problem as an illustrative example and focus on the design of modularized systems that consist of three learning modules: representation learning, feature learning and decision learning. We discuss ways to adjust each module so that the design is robust with respect to different training sample numbers. Based on these ideas, we propose two families of learning systems. One adopts the classical histogram of oriented gradients (HOG) features while the other uses successive-subspace-learning (SSL) features. We test their performance against LeNet-5, which is an end-to-end optimized neural network, for MNIST and Fashion-MNIST datasets. The number of training samples per image class goes from the extremely weak supervision condition (i.e., 1 labeled sample per class) to the strong supervision condition (i.e., 4096 labeled sample per class) with gradual transition in between (i.e., 2n, n=0, 1, cdots, 12). Experimental results show that the two families of modularized learning systems have more robust performance than LeNet-5. They both outperform LeNet-5 by a large margin for small n and have performance comparable with that of LeNet-5 for large n.