# Attention to Mean-Fields for Particle Cloud Generation

arXiv: https://arxiv.org/abs/2305.15254 · HTML: https://mlsanity.com/paper/2305.15254/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 25 references that are in the map.
- First posted on arXiv 24 May 2023.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Language Modeling with Gated Convolutional Networks](https://mlsanity.com/paper/1612.08083/) (2016)
- [Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks](https://mlsanity.com/paper/1810.00825/) (2018)
- [A Living Review of Machine Learning for Particle Physics](https://mlsanity.com/paper/2102.02770/) (2021)
- [Particle Cloud Generation with Message Passing Generative Adversarial Networks](https://mlsanity.com/paper/2106.11535/) (2021)
- [PC-JeDi: Diffusion for Particle Cloud Generation in High Energy Physics](https://mlsanity.com/paper/2303.05376/) (2023)
- [EPiC-GAN: Equivariant Point Cloud Generation for Particle Jets](https://mlsanity.com/paper/2301.08128/) (2023)
- [On the Effects of Batch and Weight Normalization in Generative Adversarial Networks](https://mlsanity.com/paper/1704.03971/) (2017)

## Built on by (most cited first, 12 of 12)

- [Masked Particle Modeling on Sets: Towards Self-Supervised High Energy Physics Foundation Models](https://mlsanity.com/paper/2401.13537/) (2024)
- [CaloClouds II: Ultra-Fast Geometry-Independent Highly-Granular Calorimeter Simulation](https://mlsanity.com/paper/2309.05704/) (2023)
- [Deep Generative Models for Detector Signature Simulation: A Taxonomic Review](https://mlsanity.com/paper/2312.09597/) (2023)
- [EPiC-ly Fast Particle Cloud Generation with Flow-Matching and Diffusion](https://mlsanity.com/paper/2310.00049/) (2023)
- [Convolutional L2LFlows: Generating Accurate Showers in Highly Granular Calorimeters Using Convolutional Normalizing Flows](https://mlsanity.com/paper/2405.20407/) (2024)
- [PC-Droid: Faster diffusion and improved quality for particle cloud generation](https://mlsanity.com/paper/2307.06836/) (2023)
- [Flow Matching Beyond Kinematics: Generating Jets with Particle-ID and Trajectory Displacement Information](https://mlsanity.com/paper/2312.00123/) (2023)
- [PIPPIN: Generating variable length full events from partons](https://mlsanity.com/paper/2406.13074/) (2024)
- [Scaling Up Diffusion and Flow-based XGBoost Models](https://mlsanity.com/paper/2408.16046/) (2024)
- [Induced Generative Adversarial Particle Transformers](https://mlsanity.com/paper/2312.04757/) (2023)
- [Jet Discrimination with Quantum Complete Graph Neural Network](https://mlsanity.com/paper/2403.04990/) (2024)
- [DeepTreeGANv2: Iterative Pooling of Point Clouds](https://mlsanity.com/paper/2312.00042/) (2023)

## Abstract (arXiv)

The generation of collider data using machine learning has emerged as a prominent research topic in particle physics due to the increasing computational challenges associated with traditional Monte Carlo simulation methods, particularly for future colliders with higher luminosity. Although generating particle clouds is analogous to generating point clouds, accurately modelling the complex correlations between the particles presents a considerable challenge. Additionally, variable particle cloud sizes further exacerbate these difficulties, necessitating more sophisticated models. In this work, we propose a novel model that utilizes an attention-based aggregation mechanism to address these challenges. The model is trained in an adversarial training paradigm, ensuring that both the generator and critic exhibit permutation equivariance/invariance with respect to their input. A novel feature matching loss in the critic is introduced to stabilize the training. The proposed model performs competitively to the state-of-art whilst having significantly fewer parameters.