# Regularized Deep Linear Discriminant Analysis

arXiv: https://arxiv.org/abs/2105.07129 · HTML: https://mlsanity.com/paper/2105.07129/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 18 references that are in the map.
- First posted on arXiv 15 May 2021.

## Builds on (most cited first, 3 of 3)

- [On Deep Multi-View Representation Learning: Objectives and Optimization](https://mlsanity.com/paper/1602.01024/) (2016)
- [Deep Linear Discriminant Analysis](https://mlsanity.com/paper/1511.04707/) (2015)
- [Fisher and Kernel Fisher Discriminant Analysis: Tutorial](https://mlsanity.com/paper/1906.09436/) (2019)

## Built on by (most cited first, 2 of 2)

- [A Deep Positive-Negative Prototype Approach to Integrated Prototypical Discriminative Learning](https://mlsanity.com/paper/2501.02477/) (2025)
- [Simplex Deep Linear Discriminant Analysis](https://mlsanity.com/paper/2601.01679/) (2026)

## Abstract (arXiv)

As a non-linear extension of the classic Linear Discriminant Analysis(LDA), Deep Linear Discriminant Analysis(DLDA) replaces the original Categorical Cross Entropy(CCE) loss function with eigenvalue-based loss function to make a deep neural network(DNN) able to learn linearly separable hidden representations. In this paper, we first point out DLDA focuses on training the cooperative discriminative ability of all the dimensions in the latent subspace, while put less emphasis on training the separable capacity of single dimension. To improve DLDA, a regularization method on within-class scatter matrix is proposed to strengthen the discriminative ability of each dimension, and also keep them complement each other. Experiment results on STL-10, CIFAR-10 and Pediatric Pneumonic Chest X-ray Dataset showed that our proposed regularization method Regularized Deep Linear Discriminant Analysis(RDLDA) outperformed DLDA and conventional neural network with CCE as objective. To further improve the discriminative ability of RDLDA in the local space, an algorithm named Subclass RDLDA is also proposed.