# Fisher and Kernel Fisher Discriminant Analysis: Tutorial

arXiv: https://arxiv.org/abs/1906.09436 · HTML: https://mlsanity.com/paper/1906.09436/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Jun 2019.

## Built on by (most cited first, 12 of 12)

- [Reproducing Kernel Hilbert Space, Mercer's Theorem, Eigenfunctions, Nystr"om Method, and Use of Kernels in Machine Learning: Tutorial and Survey](https://mlsanity.com/paper/2106.08443/) (2021)
- [ETran: Energy-Based Transferability Estimation](https://mlsanity.com/paper/2308.02027/) (2023)
- [Spectral, Probabilistic, and Deep Metric Learning: Tutorial and Survey](https://mlsanity.com/paper/2201.09267/) (2022)
- [Laplacian-Based Dimensionality Reduction Including Spectral Clustering, Laplacian Eigenmap, Locality Preserving Projection, Graph Embedding, and Diffusion Map: Tutorial and Survey](https://mlsanity.com/paper/2106.02154/) (2021)
- [Johnson-Lindenstrauss Lemma, Linear and Nonlinear Random Projections, Random Fourier Features, and Random Kitchen Sinks: Tutorial and Survey](https://mlsanity.com/paper/2108.04172/) (2021)
- [Regularized Deep Linear Discriminant Analysis](https://mlsanity.com/paper/2105.07129/) (2021)
- [Effects of Parametric and Non-Parametric Methods on High Dimensional Sparse Matrix Representations](https://mlsanity.com/paper/2202.02894/) (2022)
- [Affective Manifolds: Modeling Machine's Mind to Like, Dislike, Enjoy, Suffer, Worry, Fear, and Feel Like A Human](https://mlsanity.com/paper/2208.13386/) (2022)
- [Gravitational Dimensionality Reduction Using Newtonian Gravity and Einstein's General Relativity](https://mlsanity.com/paper/2211.01369/) (2022)
- [Pre-Trained Model Recommendation for Downstream Fine-tuning](https://mlsanity.com/paper/2403.06382/) (2024)
- [Causal vs. Anticausal merging of predictors](https://mlsanity.com/paper/2501.08426/) (2025)
- [Meta-learning One-class Classifiers with Eigenvalue Solvers for Supervised Anomaly Detection](https://mlsanity.com/paper/2103.00684/) (2021)

## Abstract (arXiv)

This is a detailed tutorial paper which explains the Fisher discriminant Analysis (FDA) and kernel FDA. We start with projection and reconstruction. Then, one- and multi-dimensional FDA subspaces are covered. Scatters in two- and then multi-classes are explained in FDA. Then, we discuss on the rank of the scatters and the dimensionality of the subspace. A real-life example is also provided for interpreting FDA. Then, possible singularity of the scatter is discussed to introduce robust FDA. PCA and FDA directions are also compared. We also prove that FDA and linear discriminant analysis are equivalent. Fisher forest is also introduced as an ensemble of fisher subspaces useful for handling data with different features and dimensionality. Afterwards, kernel FDA is explained for both one- and multi-dimensional subspaces with both two- and multi-classes. Finally, some simulations are performed on AT&T face dataset to illustrate FDA and compare it with PCA.