# Optimal Projections for Discriminative Dictionary Learning using the JL-lemma

arXiv: https://arxiv.org/abs/2308.13991 · HTML: https://mlsanity.com/paper/2308.13991/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 48 references that are in the map.
- First posted on arXiv 27 Aug 2023.

## Builds on (most cited first, 6 of 6)

- [Restricted Isometry Property of Gaussian Random Projection for Finite Set of Subspaces](https://mlsanity.com/paper/1704.02109/) (2017)
- [Nonlinear Supervised Dimensionality Reduction via Smooth Regular Embeddings](https://mlsanity.com/paper/1710.07120/) (2017)
- [Telugu OCR Framework using Deep Learning](https://mlsanity.com/paper/1509.05962/) (2015)
- [Rigorous Restricted Isometry Property of Low-Dimensional Subspaces](https://mlsanity.com/paper/1801.10058/) (2018)
- [Unraveling the Veil of Subspace RIP Through Near-Isometry on Subspaces](https://mlsanity.com/paper/1905.09608/) (2019)
- [Discriminative Dictionary Learning based on Statistical Methods](https://mlsanity.com/paper/2111.09027/) (2021)

## Abstract (arXiv)

Dimensionality reduction-based dictionary learning methods in the literature have often used iterative random projections. The dimensionality of such a random projection matrix is a random number that might not lead to a separable subspace structure in the transformed space. The convergence of such methods highly depends on the initial seed values used. Also, gradient descent-based updates might result in local minima. This paper proposes a constructive approach to derandomize the projection matrix using the Johnson-Lindenstrauss lemma. Rather than reducing dimensionality via random projections, a projection matrix derived from the proposed Modified Supervised PC analysis is used. A heuristic is proposed to decide the data perturbation levels and the dictionary atom's corresponding suitable description length. The projection matrix is derived in a single step, provides maximum feature-label consistency of the transformed space, and preserves the geometry of the original data. The projection matrix thus constructed is proved to be a JL-embedding. Despite confusing classes in the OCR datasets, the dictionary trained in the transformed space generates discriminative sparse coefficients with reduced complexity. Empirical study demonstrates that the proposed method performs well even when the number of classes and dimensionality increase. Experimentation on OCR and face recognition datasets shows better classification performance than other algorithms.