# On Data-Driven Saak Transform

arXiv: https://arxiv.org/abs/1710.04176 · HTML: https://mlsanity.com/paper/1710.04176/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Oct 2017.

## Built on by (most cited first, 22 of 22)

- [Green Learning: Introduction, Examples and Outlook](https://mlsanity.com/paper/2210.00965/) (2022)
- [On Supervised Feature Selection from High Dimensional Feature Spaces](https://mlsanity.com/paper/2203.11924/) (2022)
- [Successive Subspace Learning: An Overview](https://mlsanity.com/paper/2103.00121/) (2021)
- [E-PixelHop: An Enhanced PixelHop Method for Object Classification](https://mlsanity.com/paper/2107.02966/) (2021)
- [GSIP: Green Semantic Segmentation of Large-Scale Indoor Point Clouds](https://mlsanity.com/paper/2109.11835/) (2021)
- [PAGER: Progressive Attribute-Guided Extendable Robust Image Generation](https://mlsanity.com/paper/2206.00162/) (2022)
- [Bridging Gap between Image Pixels and Semantics via Supervision: A Survey](https://mlsanity.com/paper/2107.13757/) (2021)
- [Subspace Learning Machine (SLM): Methodology and Performance](https://mlsanity.com/paper/2205.05296/) (2022)
- [GENHOP: An Image Generation Method Based on Successive Subspace Learning](https://mlsanity.com/paper/2210.03689/) (2022)
- [Analysis of function approximation and stability of general DNNs in directed acyclic graphs using un-rectifying analysis](https://mlsanity.com/paper/2206.05997/) (2022)
- [Design of Supervision-Scalable Learning Systems: Methodology and Performance Benchmarking](https://mlsanity.com/paper/2206.09061/) (2022)
- [Enhancing Edge Intelligence with Highly Discriminant LNT Features](https://mlsanity.com/paper/2312.14968/) (2023)
- [Learn to Enhance the Negative Information in Convolutional Neural Network](https://mlsanity.com/paper/2306.10536/) (2023)
- [Statistical Attention Localization (SAL): Methodology and Application to Object Classification](https://mlsanity.com/paper/2208.01823/) (2022)
- [PointFlowHop: Green and Interpretable Scene Flow Estimation from Consecutive Point Clouds](https://mlsanity.com/paper/2302.14193/) (2023)
- [Green Steganalyzer: A Green Learning Approach to Image Steganalysis](https://mlsanity.com/paper/2306.04008/) (2023)
- [Learning DNN networks using un-rectifying ReLU with compressed sensing application](https://mlsanity.com/paper/2101.06940/) (2021)
- [LSR: A Light-Weight Super-Resolution Method](https://mlsanity.com/paper/2302.13596/) (2023)
- [A Tiny Machine Learning Model for Point Cloud Object Classification](https://mlsanity.com/paper/2303.10898/) (2023)
- [GWPT: A Green Word-Embedding-based POS Tagger](https://mlsanity.com/paper/2401.07475/) (2024)
- [Efficient Human-Object-Interaction (EHOI) Detection via Interaction Label Coding and Conditional Decision](https://mlsanity.com/paper/2408.07018/) (2024)
- [On filter design in deep convolutional neural network](https://mlsanity.com/paper/2410.21644/) (2024)

## Abstract (arXiv)

Being motivated by the multilayer RECOS (REctified-COrrelations on a Sphere) transform, we develop a data-driven Saak (Subspace approximation with augmented kernels) transform in this work. The Saak transform consists of three steps: 1) building the optimal linear subspace approximation with orthonormal bases using the second-order statistics of input vectors, 2) augmenting each transform kernel with its negative, 3) applying the rectified linear unit (ReLU) to the transform output. The Karhunen-Lo'eve transform (KLT) is used in the first step. The integration of Steps 2 and 3 is powerful since they resolve the sign confusion problem, remove the rectification loss and allow a straightforward implementation of the inverse Saak transform at the same time. Multiple Saak transforms are cascaded to transform images of a larger size. All Saak transform kernels are derived from the second-order statistics of input random vectors in a one-pass feedforward manner. Neither data labels nor backpropagation is used in kernel determination. Multi-stage Saak transforms offer a family of joint spatial-spectral representations between two extremes; namely, the full spatial-domain representation and the full spectral-domain representation. We select Saak coefficients of higher discriminant power to form a feature vector for pattern recognition, and use the MNIST dataset classification problem as an illustrative example.