# Anomalies, Representations, and Self-Supervision

arXiv: https://arxiv.org/abs/2301.04660 · HTML: https://mlsanity.com/paper/2301.04660/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 76 references that are in the map.
- First posted on arXiv 11 Jan 2023.

## Builds on (most cited first, 29 of 29)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances](https://mlsanity.com/paper/2007.08176/) (2020)
- [SSD: A Unified Framework for Self-Supervised Outlier Detection](https://mlsanity.com/paper/2103.12051/) (2021)
- [Particle Transformer for Jet Tagging](https://mlsanity.com/paper/2202.03772/) (2022)
- [Mean-Shifted Contrastive Loss for Anomaly Detection](https://mlsanity.com/paper/2106.03844/) (2021)
- [Anomaly Detection with Density Estimation](https://mlsanity.com/paper/2001.04990/) (2020)
- [ParticleNet: Jet Tagging via Particle Clouds](https://mlsanity.com/paper/1902.08570/) (2019)
- [Anomaly Detection With Conditional Variational Autoencoders](https://mlsanity.com/paper/2010.05531/) (2020)
- [Classification without labels: Learning from mixed samples in high energy physics](https://mlsanity.com/paper/1708.02949/) (2017)
- [Variational Autoencoders for New Physics Mining at the Large Hadron Collider](https://mlsanity.com/paper/1811.10276/) (2018)
- [Variational Autoencoders for Anomalous Jet Tagging](https://mlsanity.com/paper/2007.01850/) (2020)
- [The Dark Machines Anomaly Score Challenge: Benchmark Data and Model Independent Event Classification for the Large Hadron Collider](https://mlsanity.com/paper/2105.14027/) (2021)
- [Simulation Assisted Likelihood-free Anomaly Detection](https://mlsanity.com/paper/2001.05001/) (2020)
- [PELICAN: Permutation Equivariant and Lorentz Invariant or Covariant Aggregator Network for Particle Physics](https://mlsanity.com/paper/2211.00454/) (2022)
- [Self-Supervised Anomaly Detection in Computer Vision and Beyond: A Survey and Outlook](https://mlsanity.com/paper/2205.05173/) (2022)
- [Better Latent Spaces for Better Autoencoders](https://mlsanity.com/paper/2104.08291/) (2021)
- [Bump Hunting in Latent Space](https://mlsanity.com/paper/2103.06595/) (2021)
- [Challenges for Unsupervised Anomaly Detection in Particle Physics](https://mlsanity.com/paper/2110.06948/) (2021)
- [Online-compatible Unsupervised Non-resonant Anomaly Detection](https://mlsanity.com/paper/2111.06417/) (2021)
- [Lorentz group equivariant autoencoders](https://mlsanity.com/paper/2212.07347/) (2022)
- [Self-supervised similarity search for large scientific datasets](https://mlsanity.com/paper/2110.13151/) (2021)
- [Particle Graph Autoencoders and Differentiable, Learned Energy Mover's Distance](https://mlsanity.com/paper/2111.12849/) (2021)
- [Autoencoders for Semivisible Jet Detection](https://mlsanity.com/paper/2112.02864/) (2021)
- [Mining for Strong Gravitational Lenses with Self-supervised Learning](https://mlsanity.com/paper/2110.00023/) (2021)
- [IRC-safe Graph Autoencoder for unsupervised anomaly detection](https://mlsanity.com/paper/2204.12231/) (2022)
- [Null Hypothesis Test for Anomaly Detection](https://mlsanity.com/paper/2210.02226/) (2022)
- [Estimating Galactic Distances From Images Using Self-supervised Representation Learning](https://mlsanity.com/paper/2101.04293/) (2021)
- [Invariant Representation Driven Neural Classifier for Anti-QCD Jet Tagging](https://mlsanity.com/paper/2201.07199/) (2022)
- [Resonant Anomaly Detection with Multiple Reference Datasets](https://mlsanity.com/paper/2212.10579/) (2022)

## Built on by (most cited first, 11 of 11)

- [Masked Particle Modeling on Sets: Towards Self-Supervised High Energy Physics Foundation Models](https://mlsanity.com/paper/2401.13537/) (2024)
- [OmniJet-α: The first cross-task foundation model for particle physics](https://mlsanity.com/paper/2403.05618/) (2024)
- [High-dimensional and Permutation Invariant Anomaly Detection](https://mlsanity.com/paper/2306.03933/) (2023)
- [CURTAINs Flows For Flows: Constructing Unobserved Regions with Maximum Likelihood Estimation](https://mlsanity.com/paper/2305.04646/) (2023)
- [Fine-tuning machine-learned particle-flow reconstruction for new detector geometries in future colliders](https://mlsanity.com/paper/2503.00131/) (2025)
- [EveNet: A Foundation Model for Particle Collision Data Analysis](https://mlsanity.com/paper/2601.17126/) (2026)
- [Improving the performance of weak supervision searches using transfer and meta-learning](https://mlsanity.com/paper/2312.06152/) (2023)
- [DLScanner: A parameter space scanner package assisted by deep learning methods](https://mlsanity.com/paper/2412.19675/) (2024)
- [Detecting Localized Density Anomalies in Multivariate Data via Coin-Flip Statistics](https://mlsanity.com/paper/2503.23927/) (2025)
- [Contrastive Learning for Interpretable Anomaly Detection at Collider Experiments](https://mlsanity.com/paper/2608.13652/) (2026)
- [Local Diagnostics of Learned Event Representations for Anomaly Detection at the HL-LHC](https://mlsanity.com/paper/2610.13437/) (2026)

## Abstract (arXiv)

We develop a self-supervised method for density-based anomaly detection using contrastive learning, and test it using event-level anomaly data from CMS ADC2021. The AnomalyCLR technique is data-driven and uses augmentations of the background data to mimic non-Standard-Model events in a model-agnostic way. It uses a permutation-invariant Transformer Encoder architecture to map the objects measured in a collider event to the representation space, where the data augmentations define a representation space which is sensitive to potential anomalous features. An AutoEncoder trained on background representations then computes anomaly scores for a variety of signals in the representation space. With AnomalyCLR we find significant improvements on performance metrics for all signals when compared to the raw data baseline.