# Deep learning probability flows and entropy production rates in active matter

arXiv: https://arxiv.org/abs/2309.12991 · HTML: https://mlsanity.com/paper/2309.12991/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 46 references that are in the map.
- First posted on arXiv 22 Sep 2023.

## Builds on (most cited first, 10 of 10)

- [Quantifying Attention Flow in Transformers](https://mlsanity.com/paper/2005.00928/) (2020)
- [The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems](https://mlsanity.com/paper/1710.00211/) (2017)
- [Deep learning for universal linear embeddings of nonlinear dynamics](https://mlsanity.com/paper/1712.09707/) (2017)
- [Machine learning accelerated computational fluid dynamics](https://mlsanity.com/paper/2102.01010/) (2021)
- [Machine Learning for Fluid Mechanics](https://mlsanity.com/paper/1905.11075/) (2019)
- [Deep neural network solution of the electronic Schr"odinger equation](https://mlsanity.com/paper/1909.08423/) (2019)
- [Solving high-dimensional Hamilton-Jacobi-Bellman PDEs using neural networks: perspectives from the theory of controlled diffusions and measures on path space](https://mlsanity.com/paper/2005.05409/) (2020)
- [Algorithms for Solving High Dimensional PDEs: From Nonlinear Monte Carlo to Machine Learning](https://mlsanity.com/paper/2008.13333/) (2020)
- [OrbNet: Deep Learning for Quantum Chemistry Using Symmetry-Adapted Atomic-Orbital Features](https://mlsanity.com/paper/2007.08026/) (2020)
- [Probability flow solution of the Fokker-Planck equation](https://mlsanity.com/paper/2206.04642/) (2022)

## Built on by (most cited first, 15 of 15)

- [SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers](https://mlsanity.com/paper/2401.08740/) (2024)
- [A mathematical perspective on Transformers](https://mlsanity.com/paper/2312.10794/) (2023)
- [Conditioning Diffusions Using Malliavin Calculus](https://mlsanity.com/paper/2504.03461/) (2025)
- [Inferring biological processes with intrinsic noise from cross-sectional data](https://mlsanity.com/paper/2410.07501/) (2024)
- [What's the score? Automated Denoising Score Matching for Nonlinear Diffusions](https://mlsanity.com/paper/2407.07998/) (2024)
- [A brief review of the Deep BSDE method for solving high-dimensional partial differential equations](https://mlsanity.com/paper/2505.17032/) (2025)
- [Weak Generative Sampler to Efficiently Sample Invariant Distribution of Stochastic Differential Equation](https://mlsanity.com/paper/2405.19256/) (2024)
- [Identifiable learning of dissipative dynamics](https://mlsanity.com/paper/2510.24160/) (2025)
- [Score-fPINN: Fractional Score-Based Physics-Informed Neural Networks for High-Dimensional Fokker-Planck-Levy Equations](https://mlsanity.com/paper/2406.11676/) (2024)
- [Learning Stochastic Thermodynamics Directly from Correlation and Trajectory-Fluctuation Currents](https://mlsanity.com/paper/2504.19007/) (2025)
- [Inferring entropy production in many-body systems using nonequilibrium maximum entropy](https://mlsanity.com/paper/2505.10444/) (2025)
- [Radon--Wasserstein Gradient Flows for Interacting-Particle Sampling in High Dimensions](https://mlsanity.com/paper/2602.05227/) (2026)
- [MGD: Moment Guided Diffusion for Maximum Entropy Generation](https://mlsanity.com/paper/2602.17211/) (2026)
- [First-Order Trajectory Matching: Fast Ensemble Predictions of Chaotic, Turbulent, Stochastic Systems](https://mlsanity.com/paper/2606.11138/) (2026)
- [Reactive Flux Matching: Mechanism Discovery and Adaptive Sampling of Rare Events](https://mlsanity.com/paper/2606.06295/) (2026)

## Abstract (arXiv)

Active matter systems, from self-propelled colloids to motile bacteria, are characterized by the conversion of free energy into useful work at the microscopic scale. They involve physics beyond the reach of equilibrium statistical mechanics, and a persistent challenge has been to understand the nature of their nonequilibrium states. The entropy production rate and the probability current provide quantitative ways to do so by measuring the breakdown of time-reversal symmetry. Yet, their efficient computation has remained elusive, as they depend on the system's unknown and high-dimensional probability density. Here, building upon recent advances in generative modeling, we develop a deep learning framework to estimate the score of this density. We show that the score, together with the microscopic equations of motion, gives access to the entropy production rate, the probability current, and their decomposition into local contributions from individual particles. To represent the score, we introduce a novel, spatially-local transformer network architecture that learns high-order interactions between particles while respecting their underlying permutation symmetry. We demonstrate the broad utility and scalability of the method by applying it to several high-dimensional systems of active particles undergoing motility-induced phase separation (MIPS). We show that a single network trained on a system of 4096 particles at one packing fraction can generalize to other regions of the phase diagram, including systems with as many as 32768 particles. We use this observation to quantify the spatial structure of the departure from equilibrium in MIPS as a function of the number of particles and the packing fraction.