# Deep Reinforcement Learning of Marked Temporal Point Processes

arXiv: https://arxiv.org/abs/1805.09360 · HTML: https://mlsanity.com/paper/1805.09360/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 May 2018.

## Built on by (most cited first, 30 of 33)

- [Neural Temporal Point Processes: A Review](https://mlsanity.com/paper/2104.03528/) (2021)
- [Spatio-temporal Diffusion Point Processes](https://mlsanity.com/paper/2305.12403/) (2023)
- [Exploring Generative Neural Temporal Point Process](https://mlsanity.com/paper/2208.01874/) (2022)
- [Bellman Meets Hawkes: Model-Based Reinforcement Learning via Temporal Point Processes](https://mlsanity.com/paper/2201.12569/) (2022)
- [Long Horizon Forecasting With Temporal Point Processes](https://mlsanity.com/paper/2101.02815/) (2021)
- [Counterfactual Temporal Point Processes](https://mlsanity.com/paper/2111.07603/) (2021)
- [On the Predictive Accuracy of Neural Temporal Point Process Models for Continuous-time Event Data](https://mlsanity.com/paper/2306.17066/) (2023)
- [Automatic Integration for Spatiotemporal Neural Point Processes](https://mlsanity.com/paper/2310.06179/) (2023)
- [Neural Spectral Marked Point Processes](https://mlsanity.com/paper/2106.10773/) (2021)
- [An Empirical Study: Extensive Deep Temporal Point Process](https://mlsanity.com/paper/2110.09823/) (2021)
- [Modeling Inter-Dependence Between Time and Mark in Multivariate Temporal Point Processes](https://mlsanity.com/paper/2210.15294/) (2022)
- [SMURF-THP: Score Matching-based UnceRtainty quantiFication for Transformer Hawkes Process](https://mlsanity.com/paper/2310.16336/) (2023)
- [Non-Autoregressive Diffusion-based Temporal Point Processes for Continuous-Time Long-Term Event Prediction](https://mlsanity.com/paper/2311.01033/) (2023)
- [Distribution-Free Conformal Joint Prediction Regions for Neural Marked Temporal Point Processes](https://mlsanity.com/paper/2401.04612/) (2024)
- [Modeling Events and Interactions through Temporal Processes -- A Survey](https://mlsanity.com/paper/2303.06067/) (2023)
- [Proactive and Reactive Engagement of Artificial Intelligence Methods for Education: A Review](https://mlsanity.com/paper/2301.10231/) (2023)
- [A Variational Autoencoder for Neural Temporal Point Processes with Dynamic Latent Graphs](https://mlsanity.com/paper/2312.16083/) (2023)
- [Neural Spatiotemporal Point Processes: Trends and Challenges](https://mlsanity.com/paper/2502.09341/) (2025)
- [RLTutor: Reinforcement Learning Based Adaptive Tutoring System by Modeling Virtual Student with Fewer Interactions](https://mlsanity.com/paper/2108.00268/) (2021)
- [TEE4EHR: Transformer Event Encoder for Better Representation Learning in Electronic Health Records](https://mlsanity.com/paper/2402.06367/) (2024)
- [How Stable is Knowledge Base Knowledge?](https://mlsanity.com/paper/2211.00989/) (2022)
- [Cumulative Hazard Function Based Efficient Multivariate Temporal Point Process Learning](https://mlsanity.com/paper/2404.13663/) (2024)
- [Forecasting the 2016-2017 Central Apennines Earthquake Sequence with a Neural Point Process](https://mlsanity.com/paper/2301.09948/) (2023)
- [Score Matching-based Pseudolikelihood Estimation of Neural Marked Spatio-Temporal Point Process with Uncertainty Quantification](https://mlsanity.com/paper/2310.16310/) (2023)
- [Cumulative Distribution Function based General Temporal Point Processes](https://mlsanity.com/paper/2402.00388/) (2024)
- [EduQate: Generating Adaptive Curricula through RMABs in Education Settings](https://mlsanity.com/paper/2406.14122/) (2024)
- [Imitation Learning of Neural Spatio-Temporal Point Processes](https://mlsanity.com/paper/1906.05467/) (2019)
- [Least Squares Maximum and Weighted Generalization-Memorization Machines](https://mlsanity.com/paper/2308.16456/) (2023)
- [Amortized Network Intervention to Steer the Excitatory Point Processes](https://mlsanity.com/paper/2310.04159/) (2023)
- [Time After Time: Deep-Q Effect Estimation for Interventions on When and What to do](https://mlsanity.com/paper/2503.15890/) (2025)

## Abstract (arXiv)

In a wide variety of applications, humans interact with a complex environment by means of asynchronous stochastic discrete events in continuous time. Can we design online interventions that will help humans achieve certain goals in such asynchronous setting? In this paper, we address the above problem from the perspective of deep reinforcement learning of marked temporal point processes, where both the actions taken by an agent and the feedback it receives from the environment are asynchronous stochastic discrete events characterized using marked temporal point processes. In doing so, we define the agent's policy using the intensity and mark distribution of the corresponding process and then derive a flexible policy gradient method, which embeds the agent's actions and the feedback it receives into real-valued vectors using deep recurrent neural networks. Our method does not make any assumptions on the functional form of the intensity and mark distribution of the feedback and it allows for arbitrarily complex reward functions. We apply our methodology to two different applications in personalized teaching and viral marketing and, using data gathered from Duolingo and Twitter, we show that it may be able to find interventions to help learners and marketers achieve their goals more effectively than alternatives.