# BOOOM: Loss-Function-Agnostic Black-Box Optimization over Orthonormal Manifolds for Machine Learning and Statistical Inference

arXiv: https://arxiv.org/abs/2605.04087 · HTML: https://mlsanity.com/paper/2605.04087/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 3 of its 80 references that are in the map.
- First posted on arXiv 23 Apr 2026.

## Builds on (most cited first, 3 of 3)

- [An overview of gradient descent optimization algorithms](https://mlsanity.com/paper/1609.04747/) (2016)
- [Evading Adversarial Example Detection Defenses with Orthogonal Projected Gradient Descent](https://mlsanity.com/paper/2106.15023/) (2021)
- [O-ViT: Orthogonal Vision Transformer](https://mlsanity.com/paper/2201.12133/) (2022)

## Abstract (arXiv)

Optimization over the Stiefel manifold St(p,d), the set of p × d column-orthonormal matrices, is fundamental in statistics, machine learning, and scientific computing, yet remains challenging in the presence of non-convex, non-smooth, or black-box objectives. Existing methods largely rely on either convex relaxations or gradient-based Riemannian optimization, limiting applicability in derivative-free and highly multimodal settings. We propose textscBOOOM (Black-box Optimization Over Orthonormal Manifolds), a general-purpose framework for loss-function-agnostic optimization on St(p,d). The key idea is a global Givens rotation-based parametrization that maps the manifold to an unconstrained Euclidean angle space while preserving feasibility exactly. Building on this representation, BOOOM employs a structured, parallelizable, derivative-free search based on Recursive Modified Pattern Search, enabling systematic exploration through plane-wise rotations without requiring gradient information and facilitating escape from poor local optima. We establish a unified theoretical framework showing equivalence between angle-space and manifold optimization, transfer of stationarity, and global convergence in probability under mild conditions. Empirical results across diverse problems, including heterogeneous quadratic optimization, low-rank and sparse matrix decomposition, independent component analysis, and orthogonal joint diagonalization, among other widely studied settings, demonstrate strong performance relative to state-of-the-art methods, particularly in non-smooth and highly multimodal regimes. We further illustrate its practical utility through a novel supervised PCA formulation applied to metabolomics data in colorectal cancer.