# Orthogonality-Promoting Distance Metric Learning: Convex Relaxation and Theoretical Analysis

arXiv: https://arxiv.org/abs/1802.06014 · HTML: https://mlsanity.com/paper/1802.06014/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Feb 2018.

## Built on by (most cited first, 6 of 6)

- [Disentangled Representation with Causal Constraints for Counterfactual Fairness](https://mlsanity.com/paper/2208.09147/) (2022)
- [Disentangled Representation Learning for Causal Inference with Instruments](https://mlsanity.com/paper/2412.04641/) (2024)
- [Semi-Supervised Metric Learning: A Deep Resurrection](https://mlsanity.com/paper/2105.05061/) (2021)
- [Compressive Mahalanobis Metric Learning Adapts to Intrinsic Dimension](https://mlsanity.com/paper/2309.05751/) (2023)
- [Efficient 3D affinely equivariant CNNs with adaptive fusion of augmented spherical Fourier-Bessel bases](https://mlsanity.com/paper/2402.16825/) (2024)
- [Toward Better Generalization in Few-Shot Learning through the Meta-Component Combination](https://mlsanity.com/paper/2511.11632/) (2025)

## Abstract (arXiv)

Distance metric learning (DML), which learns a distance metric from labeled "similar" and "dissimilar" data pairs, is widely utilized. Recently, several works investigate orthogonality-promoting regularization (OPR), which encourages the projection vectors in DML to be close to being orthogonal, to achieve three effects: (1) high balancedness -- achieving comparable performance on both frequent and infrequent classes; (2) high compactness -- using a small number of projection vectors to achieve a "good" metric; (3) good generalizability -- alleviating overfitting to training data. While showing promising results, these approaches suffer three problems. First, they involve solving non-convex optimization problems where achieving the global optimal is NP-hard. Second, it lacks a theoretical understanding why OPR can lead to balancedness. Third, the current generalization error analysis of OPR is not directly on the regularizer. In this paper, we address these three issues by (1) seeking convex relaxations of the original nonconvex problems so that the global optimal is guaranteed to be achievable; (2) providing a formal analysis on OPR's capability of promoting balancedness; (3) providing a theoretical analysis that directly reveals the relationship between OPR and generalization performance. Experiments on various datasets demonstrate that our convex methods are more effective in promoting balancedness, compactness, and generalization, and are computationally more efficient, compared with the nonconvex methods.