# Synergistic eigenanalysis of covariance and Hessian matrices for enhanced binary classification

arXiv: https://arxiv.org/abs/2402.09281 · HTML: https://mlsanity.com/paper/2402.09281/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 109 references that are in the map.
- First posted on arXiv 14 Feb 2024.

## Builds on (most cited first, 8 of 8)

- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Learning Covariant Feature Detectors](https://mlsanity.com/paper/1605.01224/) (2016)
- [Supervised Dimensionality Reduction for Big Data](https://mlsanity.com/paper/1709.01233/) (2017)
- [Generalized Canonical Correlation Analysis for Classification](https://mlsanity.com/paper/1304.7981/) (2013)
- [On the Covariance-Hessian Relation in Evolution Strategies](https://mlsanity.com/paper/1806.03674/) (2018)
- [Spikebench: An open benchmark for spike train time-series classification](https://mlsanity.com/paper/1810.03855/) (2018)
- [The covariance matrix of Green's functions and its application to machine learning](https://mlsanity.com/paper/2004.06481/) (2020)
- [The AeroSonicDB (YPAD-0523) Dataset for Acoustic Detection and Classification of Aircraft](https://mlsanity.com/paper/2311.06368/) (2023)

## Abstract (arXiv)

Covariance and Hessian matrices have been analyzed separately in the literature for classification problems. However, integrating these matrices has the potential to enhance their combined power in improving classification performance. We present a novel approach that combines the eigenanalysis of a covariance matrix evaluated on a training set with a Hessian matrix evaluated on a deep learning model to achieve optimal class separability in binary classification tasks. Our approach is substantiated by formal proofs that establish its capability to maximize between-class mean distance (the concept of textitseparation) and minimize within-class variances (the concept of textitcompactness), which together define the two linear discriminant analysis (LDA) criteria, particularly under ideal data conditions such as isotropy around class means and dominant leading eigenvalues. By projecting data into the combined space of the most relevant eigendirections from both matrices, we achieve optimal class separability as per these LDA criteria. Empirical validation across neural and health datasets consistently supports our theoretical framework and demonstrates that our method outperforms established methods. Our method stands out by addressing both separation and compactness criteria, unlike PCA and the Hessian method, which predominantly emphasize one criterion each. This comprehensive approach captures intricate patterns and relationships, enhancing classification performance. Furthermore, through the utilization of both LDA criteria, our method outperforms LDA itself by leveraging higher-dimensional feature spaces, in accordance with Cover's theorem, which favors linear separability in higher dimensions. Additionally, our approach sheds light on complex DNN decision-making, rendering them comprehensible within a 2D space.