# Deep Linear Discriminant Analysis

arXiv: https://arxiv.org/abs/1511.04707 · HTML: https://mlsanity.com/paper/1511.04707/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Nov 2015.

## Built on by (most cited first, 23 of 23)

- [Cross-Domain Similarity Learning for Face Recognition in Unseen Domains](https://mlsanity.com/paper/2103.07503/) (2021)
- [ELM: Embedding and Logit Margins for Long-Tail Learning](https://mlsanity.com/paper/2204.13208/) (2022)
- [WDiscOOD: Out-of-Distribution Detection via Whitened Linear Discriminant Analysis](https://mlsanity.com/paper/2303.07543/) (2023)
- [Synaptic Plasticity Models and Bio-Inspired Unsupervised Deep Learning: A Survey](https://mlsanity.com/paper/2307.16236/) (2023)
- [Deep IDA: A Deep Learning Method for Integrative Discriminant Analysis of Multi-View Data with Feature Ranking -- An Application to COVID-19 severity](https://mlsanity.com/paper/2111.09964/) (2021)
- [Visual Detector Compression via Location-Aware Discriminant Analysis](https://mlsanity.com/paper/2509.17968/) (2025)
- [Variational Distillation for Multi-View Learning](https://mlsanity.com/paper/2206.09548/) (2022)
- [What can we Learn by Predicting Accuracy?](https://mlsanity.com/paper/2208.01358/) (2022)
- [Regularized Deep Linear Discriminant Analysis](https://mlsanity.com/paper/2105.07129/) (2021)
- [Effects of Parametric and Non-Parametric Methods on High Dimensional Sparse Matrix Representations](https://mlsanity.com/paper/2202.02894/) (2022)
- [A Deep Positive-Negative Prototype Approach to Integrated Prototypical Discriminative Learning](https://mlsanity.com/paper/2501.02477/) (2025)
- [Continual learning for surface defect segmentation by subnetwork creation and selection](https://mlsanity.com/paper/2312.05100/) (2023)
- [Learning a Discriminant Latent Space with Neural Discriminant Analysis](https://mlsanity.com/paper/2107.06209/) (2021)
- [Deep Linear Discriminant Analysis with Variation for Polycystic Ovary Syndrome Classification](https://mlsanity.com/paper/2303.14401/) (2023)
- [FEMDA: a unified framework for discriminant analysis](https://mlsanity.com/paper/2311.07518/) (2023)
- [Linear Discriminant Analysis with Gradient Optimization](https://mlsanity.com/paper/2506.06845/) (2025)
- [The distribution of calibrated likelihood functions on the probability-likelihood Aitchison simplex](https://mlsanity.com/paper/2509.03365/) (2025)
- [Deep Linear Discriminant Analysis Revisited](https://mlsanity.com/paper/2601.01619/) (2026)
- [Simplex Deep Linear Discriminant Analysis](https://mlsanity.com/paper/2601.01679/) (2026)
- [Implicit Bias in Deep Linear Discriminant Analysis](https://mlsanity.com/paper/2603.02622/) (2026)
- [Supervised Dimensionality Reduction Revisited: Why LDA on Frozen CNN Features Deserves a Second Look](https://mlsanity.com/paper/2604.03928/) (2026)
- [Deep Image Segmentation via Discriminant Feature Learning](https://mlsanity.com/paper/2605.14609/) (2026)
- [CAENL: Collapse-Based Active Neural Learning with Objective-Aligned Representation Control](https://mlsanity.com/paper/2610.17509/) (2026)

## Abstract (arXiv)

We introduce Deep Linear Discriminant Analysis (DeepLDA) which learns linearly separable latent representations in an end-to-end fashion. Classic LDA extracts features which preserve class separability and is used for dimensionality reduction for many classification problems. The central idea of this paper is to put LDA on top of a deep neural network. This can be seen as a non-linear extension of classic LDA. Instead of maximizing the likelihood of target labels for individual samples, we propose an objective function that pushes the network to produce feature distributions which: (a) have low variance within the same class and (b) high variance between different classes. Our objective is derived from the general LDA eigenvalue problem and still allows to train with stochastic gradient descent and back-propagation. For evaluation we test our approach on three different benchmark datasets (MNIST, CIFAR-10 and STL-10). DeepLDA produces competitive results on MNIST and CIFAR-10 and outperforms a network trained with categorical cross entropy (same architecture) on a supervised setting of STL-10.