# Pareto-optimal quantum kernel selection for unsupervised anomaly detection on real malware beaconing data

arXiv: https://arxiv.org/abs/2610.09717 · HTML: https://mlsanity.com/paper/2610.09717/

## Key facts

- No citing papers in the map yet (data through 7 Oct 2026).
- Builds on 2 of its 47 references that are in the map.
- First posted on arXiv 7 Oct 2026.

## Builds on (most cited first, 2 of 2)

- [Supervised quantum machine learning models are kernel methods](https://mlsanity.com/paper/2101.11020/) (2021)
- [Tree-Structured Parzen Estimator: Understanding Its Algorithm Components and Their Roles for Better Empirical Performance](https://mlsanity.com/paper/2304.11127/) (2023)

## Abstract (arXiv)

Quantum kernel methods are leading candidates for a practical quantum advantage in machine learning, but assessing that potential requires two quantities usually reported separately: how well a kernel performs on the task, and how far its geometry departs from the classical kernels available for the same problem. We introduce a fully unsupervised, multi-objective protocol that optimises simultaneously the normalised pseudo discrepancy (NPD), a label-free proxy for anomaly detection quality, and the geometric difference (GD) to a tuned classical reference kernel, selecting models from the resulting Pareto front. We apply it to malware beaconing detection in real network traffic, using a one-class support vector machine with fidelity and projected quantum kernels over four data encodings, on simulators and on IQM's 20-qubit Garnet processor. NPD-guided selection alone finds a fidelity kernel that beats the tuned classical baseline, but with a geometric difference too small to certify the gain as quantum. Projected kernels reach far larger geometric differences; the Pareto-selected one only marginally exceeds the baseline (AUC 0.782 versus 0.765, gC→ Q≈ 89>√N relative to that reference kernel), still below the NPD-selected fidelity kernel (0.840).