# Evaluating generative models in high energy physics

arXiv: https://arxiv.org/abs/2211.10295 · HTML: https://mlsanity.com/paper/2211.10295/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 75 references that are in the map.
- First posted on arXiv 18 Nov 2022.

## Builds on (most cited first, 22 of 22)

- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [Improved Precision and Recall Metric for Assessing Generative Models](https://mlsanity.com/paper/1904.06991/) (2019)
- [Reliable Fidelity and Diversity Metrics for Generative Models](https://mlsanity.com/paper/2002.09797/) (2020)
- [Revisiting Classifier Two-Sample Tests](https://mlsanity.com/paper/1610.06545/) (2016)
- [On Wasserstein Two Sample Testing and Related Families of Nonparametric Tests](https://mlsanity.com/paper/1509.02237/) (2015)
- [Effectively Unbiased FID and Inception Score and where to find them](https://mlsanity.com/paper/1911.07023/) (2019)
- [Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks](https://mlsanity.com/paper/1810.00825/) (2018)
- [Learning Deep Kernels for Non-Parametric Two-Sample Tests](https://mlsanity.com/paper/2002.09116/) (2020)
- [ParticleNet: Jet Tagging via Particle Clouds](https://mlsanity.com/paper/1902.08570/) (2019)
- [Accelerating Science with Generative Adversarial Networks: An Application to 3D Particle Showers in Multi-Layer Calorimeters](https://mlsanity.com/paper/1705.02355/) (2017)
- [Score-based Generative Models for Calorimeter Shower Simulation](https://mlsanity.com/paper/2206.11898/) (2022)
- [Learning Particle Physics by Example: Location-Aware Generative Adversarial Networks for Physics Synthesis](https://mlsanity.com/paper/1701.05927/) (2017)
- [CaloFlow: Fast and Accurate Generation of Calorimeter Showers with Normalizing Flows](https://mlsanity.com/paper/2106.05285/) (2021)
- [Particle Cloud Generation with Message Passing Generative Adversarial Networks](https://mlsanity.com/paper/2106.11535/) (2021)
- [Calorimetry with Deep Learning: Particle Simulation and Reconstruction for Collider Physics](https://mlsanity.com/paper/1912.06794/) (2019)
- [Metrizing Weak Convergence with Maximum Mean Discrepancies](https://mlsanity.com/paper/2006.09268/) (2020)
- [LHC analysis-specific datasets with Generative Adversarial Networks](https://mlsanity.com/paper/1901.05282/) (2019)
- [Lund jet images from generative and cycle-consistent adversarial networks](https://mlsanity.com/paper/1909.01359/) (2019)
- [Particle-based Fast Jet Simulation at the LHC with Variational Autoencoders](https://mlsanity.com/paper/2203.00520/) (2022)
- [A survey of machine learning-based physics event generation](https://mlsanity.com/paper/2106.00643/) (2021)

## Built on by (most cited first, 30 of 31)

- [Masked Particle Modeling on Sets: Towards Self-Supervised High Energy Physics Foundation Models](https://mlsanity.com/paper/2401.13537/) (2024)
- [CaloClouds: Fast Geometry-Independent Highly-Granular Calorimeter Simulation](https://mlsanity.com/paper/2305.04847/) (2023)
- [PC-JeDi: Diffusion for Particle Cloud Generation in High Energy Physics](https://mlsanity.com/paper/2303.05376/) (2023)
- [CaloClouds II: Ultra-Fast Geometry-Independent Highly-Granular Calorimeter Simulation](https://mlsanity.com/paper/2309.05704/) (2023)
- [Deep Generative Models for Detector Signature Simulation: A Taxonomic Review](https://mlsanity.com/paper/2312.09597/) (2023)
- [Convolutional L2LFlows: Generating Accurate Showers in Highly Granular Calorimeters Using Convolutional Normalizing Flows](https://mlsanity.com/paper/2405.20407/) (2024)
- [Locality-Sensitive Hashing-Based Efficient Point Transformer with Applications in High-Energy Physics](https://mlsanity.com/paper/2402.12535/) (2024)
- [Goodness of fit by Neyman-Pearson testing](https://mlsanity.com/paper/2305.14137/) (2023)
- [Refereeing the Referees: Evaluating Two-Sample Tests for Validating Generators in Precision Sciences](https://mlsanity.com/paper/2409.16336/) (2024)
- [Flow Matching Beyond Kinematics: Generating Jets with Particle-ID and Trajectory Displacement Information](https://mlsanity.com/paper/2312.00123/) (2023)
- [AdvNF: Reducing Mode Collapse in Conditional Normalising Flows using Adversarial Learning](https://mlsanity.com/paper/2401.15948/) (2024)
- [Generalizing to new geometries with Geometry-Aware Autoregressive Models (GAAMs) for fast calorimeter simulation](https://mlsanity.com/paper/2305.11531/) (2023)
- [A Comprehensive Evaluation of Generative Models in Calorimeter Shower Simulation](https://mlsanity.com/paper/2406.12898/) (2024)
- [Conditioned quantum-assisted deep generative surrogate for particle-calorimeter interactions](https://mlsanity.com/paper/2410.22870/) (2024)
- [Comparing Generative Models with the New Physics Learning Machine](https://mlsanity.com/paper/2508.02275/) (2025)
- [Learning to Validate Generative Models: a Goodness-of-Fit Approach](https://mlsanity.com/paper/2511.09118/) (2025)
- [Induced Generative Adversarial Particle Transformers](https://mlsanity.com/paper/2312.04757/) (2023)
- [Spatially Aware Linear Transformer (SAL-T) for Particle Jet Tagging](https://mlsanity.com/paper/2510.23641/) (2025)
- [SURFing to the Fundamental Limit of Jet Tagging](https://mlsanity.com/paper/2511.15779/) (2025)
- [DeepTreeGANv2: Iterative Pooling of Point Clouds](https://mlsanity.com/paper/2312.00042/) (2023)
- [Generative Models and Statistical Validation](https://mlsanity.com/paper/2605.30453/) (2026)
- [Are We Ready for AI-Driven Discovery? AI Verification Before the Next Fundamental Physics Breakthrough](https://mlsanity.com/paper/2607.10039/) (2026)
- [Differentiable Earth Mover's Distance for Data Compression at the High-Luminosity LHC](https://mlsanity.com/paper/2306.04712/) (2023)
- [Image and Point-cloud Classification for Jet Analysis in High-Energy Physics: A survey](https://mlsanity.com/paper/2403.11934/) (2024)
- [BitHEP -- The Limits of Low-Precision ML in HEP](https://mlsanity.com/paper/2504.03387/) (2025)
- [CaloTrilogy: Toward a Breakthrough in One-Step, End-to-End, Physics-Guided Shower Generation for Modern Calorimeters](https://mlsanity.com/paper/2606.04165/) (2026)
- [Unknown Unknowns: Model Misspecification in Machine Learning for Physics](https://mlsanity.com/paper/2608.13633/) (2026)
- [Probabilistic Matching of Real and Generated Data Statistics in Generative Adversarial Networks](https://mlsanity.com/paper/2306.10943/) (2023)
- [Generating High-quality Privacy-preserving Synthetic Data](https://mlsanity.com/paper/2602.06390/) (2026)
- [Learning Standard Model structure from LHC data with Riemannian flow matching](https://mlsanity.com/paper/2607.16144/) (2026)

## Abstract (arXiv)

There has been a recent explosion in research into machine-learning-based generative modeling to tackle computational challenges for simulations in high energy physics (HEP). In order to use such alternative simulators in practice, we need well-defined metrics to compare different generative models and evaluate their discrepancy from the true distributions. We present the first systematic review and investigation into evaluation metrics and their sensitivity to failure modes of generative models, using the framework of two-sample goodness-of-fit testing, and their relevance and viability for HEP. Inspired by previous work in both physics and computer vision, we propose two new metrics, the Fr'echet and kernel physics distances (FPD and KPD, respectively), and perform a variety of experiments measuring their performance on simple Gaussian-distributed, and simulated high energy jet datasets. We find FPD, in particular, to be the most sensitive metric to all alternative jet distributions tested and recommend its adoption, along with the KPD and Wasserstein distances between individual feature distributions, for evaluating generative models in HEP. We finally demonstrate the efficacy of these proposed metrics in evaluating and comparing a novel attention-based generative adversarial particle transformer to the state-of-the-art message-passing generative adversarial network jet simulation model. The code for our proposed metrics is provided in the open source JetNet Python library.