# The Dark Machines Anomaly Score Challenge: Benchmark Data and Model Independent Event Classification for the Large Hadron Collider

arXiv: https://arxiv.org/abs/2105.14027 · HTML: https://mlsanity.com/paper/2105.14027/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 116 references that are in the map.
- First posted on arXiv 28 May 2021.

## Builds on (most cited first, 24 of 24)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [Improving Variational Inference with Inverse Autoregressive Flow](https://mlsanity.com/paper/1606.04934/) (2016)
- [Adversarial Feature Learning](https://mlsanity.com/paper/1605.09782/) (2016)
- [MADE: Masked Autoencoder for Distribution Estimation](https://mlsanity.com/paper/1502.03509/) (2015)
- [Adversarially Learned Inference](https://mlsanity.com/paper/1606.00704/) (2016)
- [Adversarially Learned Anomaly Detection](https://mlsanity.com/paper/1812.02288/) (2018)
- [Energy Flow Networks: Deep Sets for Particle Jets](https://mlsanity.com/paper/1810.05165/) (2018)
- [Sylvester Normalizing Flows for Variational Inference](https://mlsanity.com/paper/1803.05649/) (2018)
- [Anomaly Detection with Density Estimation](https://mlsanity.com/paper/2001.04990/) (2020)
- [Deep Set Prediction Networks](https://mlsanity.com/paper/1906.06565/) (2019)
- [FSPool: Learning Set Representations with Featurewise Sort Pooling](https://mlsanity.com/paper/1906.02795/) (2019)
- [Classification without labels: Learning from mixed samples in high energy physics](https://mlsanity.com/paper/1708.02949/) (2017)
- [Variational Autoencoders for New Physics Mining at the Large Hadron Collider](https://mlsanity.com/paper/1811.10276/) (2018)
- [Weakly Supervised Classification in High Energy Physics](https://mlsanity.com/paper/1702.00414/) (2017)
- [Topological Obstructions to Autoencoding](https://mlsanity.com/paper/2102.08380/) (2021)
- [Nonparametric Density Estimation for High-Dimensional Data - Algorithms and Applications](https://mlsanity.com/paper/1904.00176/) (2019)
- [Adversarially Learned Anomaly Detection on CMS Open Data: re-discovering the top quark](https://mlsanity.com/paper/2005.01598/) (2020)
- [Learning to Classify from Impure Samples with High-Dimensional Data](https://mlsanity.com/paper/1801.10158/) (2018)
- [(Machine) Learning to Do More with Less](https://mlsanity.com/paper/1706.09451/) (2017)
- [Convolutional Normalizing Flows](https://mlsanity.com/paper/1711.02255/) (2017)

## Built on by (most cited first, 29 of 29)

- [Machine Learning in the Search for New Fundamental Physics](https://mlsanity.com/paper/2112.03769/) (2021)
- [Machine Learning for Anomaly Detection in Particle Physics](https://mlsanity.com/paper/2312.14190/) (2023)
- [PC-JeDi: Diffusion for Particle Cloud Generation in High Energy Physics](https://mlsanity.com/paper/2303.05376/) (2023)
- [Learning new physics efficiently with nonparametric methods](https://mlsanity.com/paper/2204.02317/) (2022)
- [Quantum anomaly detection in the latent space of proton collision events at the LHC](https://mlsanity.com/paper/2301.10780/) (2023)
- [Deep Generative Models for Detector Signature Simulation: A Taxonomic Review](https://mlsanity.com/paper/2312.09597/) (2023)
- [Challenges for Unsupervised Anomaly Detection in Particle Physics](https://mlsanity.com/paper/2110.06948/) (2021)
- [Online-compatible Unsupervised Non-resonant Anomaly Detection](https://mlsanity.com/paper/2111.06417/) (2021)
- [Large Physics Models: Towards a collaborative approach with Large Language Models and Foundation Models](https://mlsanity.com/paper/2501.05382/) (2025)
- [High-dimensional and Permutation Invariant Anomaly Detection](https://mlsanity.com/paper/2306.03933/) (2023)
- [Anomalies, Representations, and Self-Supervision](https://mlsanity.com/paper/2301.04660/) (2023)
- [CURTAINs Flows For Flows: Constructing Unobserved Regions with Maximum Likelihood Estimation](https://mlsanity.com/paper/2305.04646/) (2023)
- [Improving new physics searches with diffusion models for event observables and jet constituents](https://mlsanity.com/paper/2312.10130/) (2023)
- [Autoencoders for Semivisible Jet Detection](https://mlsanity.com/paper/2112.02864/) (2021)
- [Unravelling physics beyond the standard model with classical and quantum anomaly detection](https://mlsanity.com/paper/2301.10787/) (2023)
- [New Methods and Datasets for Group Anomaly Detection From Fundamental Physics](https://mlsanity.com/paper/2107.02821/) (2021)
- [Benchmarking energy consumption and latency for neuromorphic computing in condensed matter and particle physics](https://mlsanity.com/paper/2209.10481/) (2022)
- [Shared Data and Algorithms for Deep Learning in Fundamental Physics](https://mlsanity.com/paper/2107.00656/) (2021)
- [Funnels: Exact maximum likelihood with dimensionality reduction](https://mlsanity.com/paper/2112.08069/) (2021)
- [Resonant Anomaly Detection with Multiple Reference Datasets](https://mlsanity.com/paper/2212.10579/) (2022)
- [Universal New Physics Latent Space](https://mlsanity.com/paper/2407.20315/) (2024)
- [ColliderML: The First Release of an OpenDataDetector High-Luminosity Physics Benchmark Dataset](https://mlsanity.com/paper/2512.15230/) (2025)
- [Model-Agnostic Signal Discovery with Machine Learning: Bridging the Gap Between Theory and Practice](https://mlsanity.com/paper/2605.31103/) (2026)
- [Decorrelation using Optimal Transport](https://mlsanity.com/paper/2307.05187/) (2023)
- [TRANSIT your events into a new mass: Fast background interpolation for weakly-supervised anomaly searches](https://mlsanity.com/paper/2503.04342/) (2025)
- [Generator Based Inference (GBI)](https://mlsanity.com/paper/2506.00119/) (2025)
- [SparsePixels: Efficient Convolution for Sparse Data on FPGAs](https://mlsanity.com/paper/2512.06208/) (2025)
- [Contrastive Learning for Interpretable Anomaly Detection at Collider Experiments](https://mlsanity.com/paper/2608.13652/) (2026)
- [Local Diagnostics of Learned Event Representations for Anomaly Detection at the HL-LHC](https://mlsanity.com/paper/2610.13437/) (2026)

## Abstract (arXiv)

We describe the outcome of a data challenge conducted as part of the Dark Machines Initiative and the Les Houches 2019 workshop on Physics at TeV colliders. The challenged aims at detecting signals of new physics at the LHC using unsupervised machine learning algorithms. First, we propose how an anomaly score could be implemented to define model-independent signal regions in LHC searches. We define and describe a large benchmark dataset, consisting of >1 Billion simulated LHC events corresponding to 10~rmfb-1 of proton-proton collisions at a center-of-mass energy of 13 TeV. We then review a wide range of anomaly detection and density estimation algorithms, developed in the context of the data challenge, and we measure their performance in a set of realistic analysis environments. We draw a number of useful conclusions that will aid the development of unsupervised new physics searches during the third run of the LHC, and provide our benchmark dataset for future studies at https://www.phenoMLdata.org. Code to reproduce the analysis is provided at https://github.com/bostdiek/DarkMachines-UnsupervisedChallenge.