# Unsupervised Energy-based Out-of-distribution Detection using Stiefel-Restricted Kernel Machine

arXiv: https://arxiv.org/abs/2102.08443 · HTML: https://mlsanity.com/paper/2102.08443/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 32 references that are in the map.
- First posted on arXiv 16 Feb 2021.

## Builds on (most cited first, 16 of 16)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks](https://mlsanity.com/paper/1807.03888/) (2018)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [Energy-based Out-of-distribution Detection](https://mlsanity.com/paper/2010.03759/) (2020)
- [Deep Anomaly Detection with Outlier Exposure](https://mlsanity.com/paper/1812.04606/) (2018)
- [Do Deep Generative Models Know What They Don't Know?](https://mlsanity.com/paper/1810.09136/) (2018)
- [Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One](https://mlsanity.com/paper/1912.03263/) (2019)
- [Efficient Riemannian Optimization on the Stiefel Manifold via the Cayley Transform](https://mlsanity.com/paper/2002.01113/) (2020)
- [Variational Recurrent Auto-Encoders](https://mlsanity.com/paper/1412.6581/) (2014)
- [Disentangled Representation Learning and Generation with Manifold Optimization](https://mlsanity.com/paper/2006.07046/) (2020)
- [Are generative deep models for novelty detection truly better?](https://mlsanity.com/paper/1807.05027/) (2018)
- [Robust Generative Restricted Kernel Machines using Weighted Conjugate Feature Duality](https://mlsanity.com/paper/2002.01180/) (2020)

## Built on by (most cited first, 11 of 11)

- [Energy-based Latent Aligner for Incremental Learning](https://mlsanity.com/paper/2203.14952/) (2022)
- [Kernel PCA for Out-of-Distribution Detection](https://mlsanity.com/paper/2402.02949/) (2024)
- [Out-of-Distribution Generalization in Time Series: A Survey](https://mlsanity.com/paper/2503.13868/) (2025)
- [Revisiting Deep Ensemble for Out-of-Distribution Detection: A Loss Landscape Perspective](https://mlsanity.com/paper/2310.14227/) (2023)
- [CRoFT: Robust Fine-Tuning with Concurrent Optimization for OOD Generalization and Open-Set OOD Detection](https://mlsanity.com/paper/2405.16417/) (2024)
- [Tensor-based Multi-view Spectral Clustering via Shared Latent Space](https://mlsanity.com/paper/2207.11559/) (2022)
- [One Class Restricted Kernel Machines](https://mlsanity.com/paper/2502.10443/) (2025)
- [Class-wise Thresholding for Robust Out-of-Distribution Detection](https://mlsanity.com/paper/2110.15292/) (2021)
- [Multi-view Kernel PCA for Time series Forecasting](https://mlsanity.com/paper/2301.09811/) (2023)
- [Duality in Multi-View Restricted Kernel Machines](https://mlsanity.com/paper/2305.17251/) (2023)
- [Combining Primal and Dual Representations in Deep Restricted Kernel Machines Classifiers](https://mlsanity.com/paper/2306.07015/) (2023)

## Abstract (arXiv)

Detecting out-of-distribution (OOD) samples is an essential requirement for the deployment of machine learning systems in the real world. Until now, research on energy-based OOD detectors has focused on the softmax confidence score from a pre-trained neural network classifier with access to class labels. In contrast, we propose an unsupervised energy-based OOD detector leveraging the Stiefel-Restricted Kernel Machine (St-RKM). Training requires minimizing an objective function with an autoencoder loss term and the RKM energy where the interconnection matrix lies on the Stiefel manifold. Further, we outline multiple energy function definitions based on the RKM framework and discuss their utility. In the experiments on standard datasets, the proposed method improves over the existing energy-based OOD detectors and deep generative models. Through several ablation studies, we further illustrate the merit of each proposed energy function on the OOD detection performance.