# Deep Kernel Principal Component Analysis for Multi-level Feature Learning

arXiv: https://arxiv.org/abs/2302.11220 · HTML: https://mlsanity.com/paper/2302.11220/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 48 references that are in the map.
- First posted on arXiv 22 Feb 2023.

## Builds on (most cited first, 16 of 16)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations](https://mlsanity.com/paper/1811.12359/) (2018)
- [InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets](https://mlsanity.com/paper/1606.03657/) (2016)
- [Disentangling by Factorising](https://mlsanity.com/paper/1802.05983/) (2018)
- [Riemannian Adaptive Optimization Methods](https://mlsanity.com/paper/1810.00760/) (2018)
- [Robustly Disentangled Causal Mechanisms: Validating Deep Representations for Interventional Robustness](https://mlsanity.com/paper/1811.00007/) (2018)
- [Theory and Evaluation Metrics for Learning Disentangled Representations](https://mlsanity.com/paper/1908.09961/) (2019)
- [On Fast Leverage Score Sampling and Optimal Learning](https://mlsanity.com/paper/1810.13258/) (2018)
- [A representer theorem for deep kernel learning](https://mlsanity.com/paper/1709.10441/) (2017)
- [Backward Feature Correction: How Deep Learning Performs Deep (Hierarchical) Learning](https://mlsanity.com/paper/2001.04413/) (2020)
- [Autoencoding any Data through Kernel Autoencoders](https://mlsanity.com/paper/1805.11028/) (2018)
- [Unsupervised learning of disentangled representations in deep restricted kernel machines with orthogonality constraints](https://mlsanity.com/paper/2011.12659/) (2020)
- [Learning Interpretable Disentangled Representations using Adversarial VAEs](https://mlsanity.com/paper/1904.08491/) (2019)
- [Generative Restricted Kernel Machines: A Framework for Multi-view Generation and Disentangled Feature Learning](https://mlsanity.com/paper/1906.08144/) (2019)
- [Diversity sampling is an implicit regularization for kernel methods](https://mlsanity.com/paper/2002.08616/) (2020)

## Built on by (most cited first, 6 of 6)

- [Convolutional Deep Kernel Machines](https://mlsanity.com/paper/2309.09814/) (2023)
- [Evaluation in EEG Emotion Recognition: State-of-the-Art Review and Unified Framework](https://mlsanity.com/paper/2505.18175/) (2025)
- [Duality in Multi-View Restricted Kernel Machines](https://mlsanity.com/paper/2305.17251/) (2023)
- [Combining Primal and Dual Representations in Deep Restricted Kernel Machines Classifiers](https://mlsanity.com/paper/2306.07015/) (2023)
- [Rethinking PCA Through Duality](https://mlsanity.com/paper/2510.18130/) (2025)
- [Rotation-Based Subspace Tracking for Robust Kernel PCA on Streaming Data](https://mlsanity.com/paper/2609.15488/) (2026)

## Abstract (arXiv)

Principal Component Analysis (PCA) and its nonlinear extension Kernel PCA (KPCA) are widely used across science and industry for data analysis and dimensionality reduction. Modern deep learning tools have achieved great empirical success, but a framework for deep principal component analysis is still lacking. Here we develop a deep kernel PCA methodology (DKPCA) to extract multiple levels of the most informative components of the data. Our scheme can effectively identify new hierarchical variables, called deep principal components, capturing the main characteristics of high-dimensional data through a simple and interpretable numerical optimization. We couple the principal components of multiple KPCA levels, theoretically showing that DKPCA creates both forward and backward dependency across levels, which has not been explored in kernel methods and yet is crucial to extract more informative features. Various experimental evaluations on multiple data types show that DKPCA finds more efficient and disentangled representations with higher explained variance in fewer principal components, compared to the shallow KPCA. We demonstrate that our method allows for effective hierarchical data exploration, with the ability to separate the key generative factors of the input data both for large datasets and when few training samples are available. Overall, DKPCA can facilitate the extraction of useful patterns from high-dimensional data by learning more informative features organized in different levels, giving diversified aspects to explore the variation factors in the data, while maintaining a simple mathematical formulation.