# Learning Temporal Point Processes via Reinforcement Learning

arXiv: https://arxiv.org/abs/1811.05016 · HTML: https://mlsanity.com/paper/1811.05016/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Nov 2018.

## Built on by (most cited first, 30 of 40)

- [Neural Temporal Point Processes: A Review](https://mlsanity.com/paper/2104.03528/) (2021)
- [Spatio-temporal Diffusion Point Processes](https://mlsanity.com/paper/2305.12403/) (2023)
- [Exploring Generative Neural Temporal Point Process](https://mlsanity.com/paper/2208.01874/) (2022)
- [On Computation and Generalization of Generative Adversarial Imitation Learning](https://mlsanity.com/paper/2001.02792/) (2020)
- [Add and Thin: Diffusion for Temporal Point Processes](https://mlsanity.com/paper/2311.01139/) (2023)
- [Automatic Integration for Spatiotemporal Neural Point Processes](https://mlsanity.com/paper/2310.06179/) (2023)
- [Neural Spectral Marked Point Processes](https://mlsanity.com/paper/2106.10773/) (2021)
- [An Empirical Study: Extensive Deep Temporal Point Process](https://mlsanity.com/paper/2110.09823/) (2021)
- [Edit-Based Flow Matching for Temporal Point Processes](https://mlsanity.com/paper/2510.06050/) (2025)
- [Modeling Inter-Dependence Between Time and Mark in Multivariate Temporal Point Processes](https://mlsanity.com/paper/2210.15294/) (2022)
- [Conditional Generative Modeling for High-dimensional Marked Temporal Point Processes](https://mlsanity.com/paper/2305.12569/) (2023)
- [SMURF-THP: Score Matching-based UnceRtainty quantiFication for Transformer Hawkes Process](https://mlsanity.com/paper/2310.16336/) (2023)
- [Distribution-Free Conformal Joint Prediction Regions for Neural Marked Temporal Point Processes](https://mlsanity.com/paper/2401.04612/) (2024)
- [Detecting Anomalous Event Sequences with Temporal Point Processes](https://mlsanity.com/paper/2106.04465/) (2021)
- [Modeling Events and Interactions through Temporal Processes -- A Survey](https://mlsanity.com/paper/2303.06067/) (2023)
- [Mamba Hawkes Process](https://mlsanity.com/paper/2407.05302/) (2024)
- [Neural Spatiotemporal Point Processes: Trends and Challenges](https://mlsanity.com/paper/2502.09341/) (2025)
- [Solar Radiation Ramping Events Modeling Using Spatio-temporal Point Processes](https://mlsanity.com/paper/2101.11179/) (2021)
- [A Temporal Kernel Approach for Deep Learning with Continuous-time Information](https://mlsanity.com/paper/2103.15213/) (2021)
- [Understanding the Spread of COVID-19 Epidemic: A Spatio-Temporal Point Process View](https://mlsanity.com/paper/2106.13097/) (2021)
- [Modeling Continuous Time Sequences with Intermittent Observations using Marked Temporal Point Processes](https://mlsanity.com/paper/2206.12414/) (2022)
- [RoTHP: Rotary Position Embedding-based Transformer Hawkes Process](https://mlsanity.com/paper/2405.06985/) (2024)
- [Scalable Marked Point Processes for Exchangeable and Non-Exchangeable Event Sequences](https://mlsanity.com/paper/2105.14574/) (2021)
- [On Non-asymptotic Theory of Recurrent Neural Networks in Temporal Point Processes](https://mlsanity.com/paper/2406.00630/) (2024)
- [EarthquakeNPP: A Benchmark for Earthquake Forecasting with Neural Point Processes](https://mlsanity.com/paper/2410.08226/) (2024)
- [Recurrent Neural Goodness-of-Fit Test for Time Series](https://mlsanity.com/paper/2410.13986/) (2024)
- [Deep Convolution for Irregularly Sampled Temporal Point Clouds](https://mlsanity.com/paper/2105.00137/) (2021)
- [Variational Neural Temporal Point Process](https://mlsanity.com/paper/2202.10585/) (2022)
- [Forecasting the 2016-2017 Central Apennines Earthquake Sequence with a Neural Point Process](https://mlsanity.com/paper/2301.09948/) (2023)
- [Score Matching-based Pseudolikelihood Estimation of Neural Marked Spatio-Temporal Point Process with Uncertainty Quantification](https://mlsanity.com/paper/2310.16310/) (2023)

## Abstract (arXiv)

Social goods, such as healthcare, smart city, and information networks, often produce ordered event data in continuous time. The generative processes of these event data can be very complex, requiring flexible models to capture their dynamics. Temporal point processes offer an elegant framework for modeling event data without discretizing the time. However, the existing maximum-likelihood-estimation (MLE) learning paradigm requires hand-crafting the intensity function beforehand and cannot directly monitor the goodness-of-fit of the estimated model in the process of training. To alleviate the risk of model-misspecification in MLE, we propose to generate samples from the generative model and monitor the quality of the samples in the process of training until the samples and the real data are indistinguishable. We take inspiration from reinforcement learning (RL) and treat the generation of each event as the action taken by a stochastic policy. We parameterize the policy as a flexible recurrent neural network and gradually improve the policy to mimic the observed event distribution. Since the reward function is unknown in this setting, we uncover an analytic and nonparametric form of the reward function using an inverse reinforcement learning formulation. This new RL framework allows us to derive an efficient policy gradient algorithm for learning flexible point process models, and we show that it performs well in both synthetic and real data.