# Out-of-Distribution Radar Detection with Complex VAEs: Theory, Whitening, and ANMF Fusion

arXiv: https://arxiv.org/abs/2601.18677 · HTML: https://mlsanity.com/paper/2601.18677/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 55 references that are in the map.
- First posted on arXiv 26 Jan 2026.

## Builds on (most cited first, 15 of 15)

- [Energy-based Out-of-distribution Detection](https://mlsanity.com/paper/2010.03759/) (2020)
- [Generalized Out-of-Distribution Detection: A Survey](https://mlsanity.com/paper/2110.11334/) (2021)
- [Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy](https://mlsanity.com/paper/2110.02642/) (2021)
- [TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Data](https://mlsanity.com/paper/2201.07284/) (2022)
- [Stochastic Normalizing Flows](https://mlsanity.com/paper/2002.06707/) (2020)
- [SurVAE Flows: Surjections to Bridge the Gap between VAEs and Flows](https://mlsanity.com/paper/2007.02731/) (2020)
- [Contrastive Out-of-Distribution Detection for Pretrained Transformers](https://mlsanity.com/paper/2104.08812/) (2021)
- [Anomaly detection through latent space restoration using vector-quantized variational autoencoders](https://mlsanity.com/paper/2012.06765/) (2020)
- [Detecting Out-of-distribution Samples via Variational Auto-encoder with Reliable Uncertainty Estimation](https://mlsanity.com/paper/2007.08128/) (2020)
- [Transformer-based out-of-distribution detection for clinically safe segmentation](https://mlsanity.com/paper/2205.10650/) (2022)
- [On Out-of-Distribution Detection for Audio with Deep Nearest Neighbors](https://mlsanity.com/paper/2210.15283/) (2022)
- [MCROOD: Multi-Class Radar Out-Of-Distribution Detection](https://mlsanity.com/paper/2303.06232/) (2023)
- [HOOD: Real-Time Human Presence and Out-of-Distribution Detection Using FMCW Radar](https://mlsanity.com/paper/2308.02396/) (2023)
- [A Deep Representation Learning-based Speech Enhancement Method Using Complex Convolution Recurrent Variational Autoencoder](https://mlsanity.com/paper/2312.09620/) (2023)
- [Out-of-Distribution Radar Detection in Compound Clutter and Thermal Noise through Variational Autoencoders](https://mlsanity.com/paper/2503.04861/) (2025)

## Abstract (arXiv)

We investigate the detection of weak complex-valued signals immersed in non-Gaussian, range-varying interference, with emphasis on maritime radar scenarios. The proposed methodology exploits a Complex-valued Variational AutoEncoder (CVAE) trained exclusively on clutter-plus-noise to perform Out-Of-Distribution detection. By operating directly on in-phase / quadrature samples, the CVAE preserves phase and Doppler structure and is assessed in two configurations: (i) using unprocessed range profiles and (ii) after local whitening, where per-range covariance estimates are obtained from neighboring profiles. Using extensive simulations together with real sea-clutter data from the CSIR maritime dataset, we benchmark performance against classical and adaptive detectors (MF, NMF, AMF-SCM, ANMF-SCM, ANMF-Tyler). In both configurations, the CVAE yields a higher detection probability Pd at matched false-alarm rate Pfa, with the most notable improvements observed under whitening. We further integrate the CVAE with the ANMF through a weighted log-p fusion rule at the decision level, attaining enhanced robustness in strongly non-Gaussian clutter and enabling empirically calibrated Pfa control under H0. Overall, the results demonstrate that statistical normalization combined with complex-valued generative modeling substantively improves detection in realistic sea-clutter conditions, and that the fused CVAE-ANMF scheme constitutes a competitive alternative to established model-based detectors.