# Sparx: Distributed Outlier Detection at Scale

arXiv: https://arxiv.org/abs/2206.01281 · HTML: https://mlsanity.com/paper/2206.01281/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jun 2022.

## Built on by (most cited first, 5 of 5)

- [ADBench: Anomaly Detection Benchmark](https://mlsanity.com/paper/2206.09426/) (2022)
- [D.MCA: Outlier Detection with Explicit Micro-Cluster Assignments](https://mlsanity.com/paper/2210.08212/) (2022)
- [From Explanation to Action: An End-to-End Human-in-the-loop Framework for Anomaly Reasoning and Management](https://mlsanity.com/paper/2304.03368/) (2023)
- [McCatch: Scalable Microcluster Detection in Dimensional and Nondimensional Datasets](https://mlsanity.com/paper/2403.08027/) (2024)
- [Quality Assessment of Tabular Data using Large Language Models and Code Generation](https://mlsanity.com/paper/2509.10572/) (2025)

## Abstract (arXiv)

There is no shortage of outlier detection (OD) algorithms in the literature, yet a vast body of them are designed for a single machine. With the increasing reality of already cloud-resident datasets comes the need for distributed OD techniques. This area, however, is not only understudied but also short of public-domain implementations for practical use. This paper aims to fill this gap: We design Sparx, a data-parallel OD algorithm suitable for shared-nothing infrastructures, which we specifically implement in Apache Spark. Through extensive experiments on three real-world datasets, with several billions of points and millions of features, we show that existing open-source solutions fail to scale up; either by large number of points or high dimensionality, whereas Sparx yields scalable and effective performance. To facilitate practical use of OD on modern-scale datasets, we open-source Sparx under the Apache license at https://tinyurl.com/sparx2022.