# Supervised Quadratic Feature Analysis: Information Geometry Approach for Dimensionality Reduction

arXiv: https://arxiv.org/abs/2502.00168 · HTML: https://mlsanity.com/paper/2502.00168/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 81 references that are in the map.
- First posted on arXiv 31 Jan 2025.

## Builds on (most cited first, 19 of 19)

- [Generalized Shape Metrics on Neural Representations](https://mlsanity.com/paper/2110.14739/) (2021)
- [Parametric UMAP embeddings for representation and semi-supervised learning](https://mlsanity.com/paper/2009.12981/) (2020)
- [Fisher-Rao Metric, Geometry, and Complexity of Neural Networks](https://mlsanity.com/paper/1711.01530/) (2017)
- [Trivializations for Gradient-Based Optimization on Manifolds](https://mlsanity.com/paper/1909.09501/) (2019)
- [An elementary introduction to information geometry](https://mlsanity.com/paper/1808.08271/) (2018)
- [Neural population geometry: An approach for understanding biological and artificial neural networks](https://mlsanity.com/paper/2104.07059/) (2021)
- [The Geometry of Deep Generative Image Models and its Applications](https://mlsanity.com/paper/2101.06006/) (2021)
- [Neural tuning and representational geometry](https://mlsanity.com/paper/2104.09743/) (2021)
- [Wasserstein Discriminant Analysis](https://mlsanity.com/paper/1608.08063/) (2016)
- [Pulling back information geometry](https://mlsanity.com/paper/2106.05367/) (2021)
- [From Manifold to Manifold: Geometry-Aware Dimensionality Reduction for SPD Matrices](https://mlsanity.com/paper/1407.1120/) (2014)
- [Representational dissimilarity metric spaces for stochastic neural networks](https://mlsanity.com/paper/2211.11665/) (2022)
- [Duality of Bures and Shape Distances with Implications for Comparing Neural Representations](https://mlsanity.com/paper/2311.11436/) (2023)
- [metric-learn: Metric Learning Algorithms in Python](https://mlsanity.com/paper/1908.04710/) (2019)
- [A numerical approximation method for the Fisher-Rao distance between multivariate normal distributions](https://mlsanity.com/paper/2302.08175/) (2023)
- [Discriminating image representations with principal distortions](https://mlsanity.com/paper/2410.15433/) (2024)
- [Perceptual Scales Predicted by Fisher Information Metrics](https://mlsanity.com/paper/2310.11759/) (2023)
- [pyBregMan: A Python library for Bregman Manifolds](https://mlsanity.com/paper/2408.04175/) (2024)
- [Discriminative Features via Generalized Eigenvectors](https://mlsanity.com/paper/1310.1934/) (2013)

## Abstract (arXiv)

Supervised dimensionality reduction maps labeled data into a low-dimensional feature space while preserving class separation. A common strategy is to learn features that maximize a measure of statistical dissimilarity between the class-conditional probability distributions. Information geometry, which is rooted in Riemannian geometry, provides an alternative framework for measuring class dissimilarity. It treats probability distributions as points in a statistical manifold and uses the Fisher information metric to define a geodesic distance--the Fisher-Rao distance--between distributions The Fisher-Rao distance is an appealing candidate for measuring class separation because the Fisher information metric is a local measure of discriminability, and because it allows a geometric interpretation. Here, we present Supervised Quadratic Feature Analysis (SQFA), a supervised dimensionality reduction method which learns linear features that maximize Fisher-Rao distances between class-conditional distributions, under Gaussian assumptions. In multiple real world datasets, we find that SQFA features support classification accuracy that is competitive with features that maximize more popular measures of dissimilarity, or that are learned by other state-of-the-art dimensionality reduction methods. Notably, the best classification accuracy is achieved by SQFA-H features, a variant of SQFA that maximizes the Hellinger distance, a rarely used objective for dimensionality reduction. These results demonstrate the potential of information geometry as a tool for supervised dimensionality reduction. We provide a Python implementation of SQFA at https://github.com/dherrera1911/sqfa.