# Stacking-Based Deep Neural Network: Deep Analytic Network for Pattern Classification

arXiv: https://arxiv.org/abs/1811.07184 · HTML: https://mlsanity.com/paper/1811.07184/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Nov 2018.

## Built on by (most cited first, 7 of 7)

- [Ensemble deep learning: A review](https://mlsanity.com/paper/2104.02395/) (2021)
- [ACIL: Analytic Class-Incremental Learning with Absolute Memorization and Privacy Protection](https://mlsanity.com/paper/2205.14922/) (2022)
- [The Art of Deception: Robust Backdoor Attack using Dynamic Stacking of Triggers](https://mlsanity.com/paper/2401.01537/) (2024)
- [An Adaptive and Stability-Promoting Layerwise Training Approach for Sparse Deep Neural Network Architecture](https://mlsanity.com/paper/2211.06860/) (2022)
- [Analytic Learning of Convolutional Neural Network For Pattern Recognition](https://mlsanity.com/paper/2202.06504/) (2022)
- [DeepAFL: Deep Analytic Federated Learning](https://mlsanity.com/paper/2603.00579/) (2026)
- [Review of AlexNet for Medical Image Classification](https://mlsanity.com/paper/2311.08655/) (2023)

## Abstract (arXiv)

Stacking-based deep neural network (S-DNN) is aggregated with pluralities of basic learning modules, one after another, to synthesize a deep neural network (DNN) alternative for pattern classification. Contrary to the DNNs trained end to end by backpropagation (BP), each S-DNN layer, i.e., a self-learnable module, is to be trained decisively and independently without BP intervention. In this paper, a ridge regression-based S-DNN, dubbed deep analytic network (DAN), along with its kernelization (K-DAN), are devised for multilayer feature re-learning from the pre-extracted baseline features and the structured features. Our theoretical formulation demonstrates that DAN/K-DAN re-learn by perturbing the intra/inter-class variations, apart from diminishing the prediction errors. We scrutinize the DAN/K-DAN performance for pattern classification on datasets of varying domains - faces, handwritten digits, generic objects, to name a few. Unlike the typical BP-optimized DNNs to be trained from gigantic datasets by GPU, we disclose that DAN/K-DAN are trainable using only CPU even for small-scale training sets. Our experimental results disclose that DAN/K-DAN outperform the present S-DNNs and also the BP-trained DNNs, including multiplayer perceptron, deep belief network, etc., without data augmentation applied.