mlsanityData to 7 Oct 2026

Bellman Meets Hawkes: Model-Based Reinforcement Learning via Temporal Point Processes

arXiv 2201.12569 ↗First posted 29 Jan 2022cs.LG cs.AIcited by 23

Not on the map. Why?

Abstract (arXiv)

We consider a sequential decision making problem where the agent faces the environment characterized by the stochastic discrete events and seeks an optimal intervention policy such that its long-term reward is maximized. This problem exists ubiquitously in social media, finance and health informatics but is rarely investigated by the conventional research in reinforcement learning. To this end, we present a novel framework of the model-based reinforcement learning where the agent's actions and observations are asynchronous stochastic discrete events occurring in continuous-time. We model the dynamics of the environment by Hawkes process with external intervention control term and develop an algorithm to embed such process in the Bellman equation which guides the direction of the value gradient. We demonstrate the superiority of our method in both synthetic simulator and real-world problem.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 32 of its 56 references that are in the map. First posted on arXiv 29 Jan 2022.

Builds on

32 of 56 references.

References · 32

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2018Neural Ordinary Differential Equationscited by 3,733
  4. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  5. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  6. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  7. 2016OpenAI Gymcited by 1,684
  8. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  9. 2018World Modelscited by 1,444
  10. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  11. 2018Self-Attention with Relative Position Representationscited by 1,112
  12. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 753
20 more
  1. 2016Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updatescited by 342
  2. 2016Safe, Multi-Agent, Reinforcement Learning for Autonomous Drivingcited by 301
  3. 2019Stabilizing Transformers for Reinforcement Learningcited by 262
  4. 2020Transformer Hawkes Processcited by 230
  5. 2015Action-Conditional Video Prediction using Deep Networks in Atari Gamescited by 171
  6. 2019Soft Actor-Critic for Discrete Action Settingscited by 161
  7. 2015Learning Continuous Control Policies by Stochastic Value Gradientscited by 156
  8. 2019Neural Jump Stochastic Differential Equationscited by 147
  9. 2018Deep Variational Reinforcement Learning for POMDPscited by 122
  10. 2020Neural Spatio-Temporal Point Processescited by 92
  11. 2015SEISMIC: A Self-Exciting Point Process Model for Predicting Tweet Popularitycited by 76
  12. 2018Lecture Notes: Temporal Point Processes and the Conditional Intensity Functioncited by 40
  13. 2021Continuous-Time Model-Based Reinforcement Learningcited by 39
  14. 2018Deep Reinforcement Learning of Marked Temporal Point Processescited by 32
  15. 2020Model-based Reinforcement Learning for Semi-Markov Decision Processes with Neural ODEscited by 32
  16. 2017Fake News Mitigation via Point Process Based Interventioncited by 24
  17. 2017A Tutorial on Hawkes Processes for Events in Social Mediacited by 15
  18. 2018Steering Social Activity: A Stochastic Optimal Control Point Of Viewcited by 6
  19. 2016Hierarchical Decision Making In Electricity Grid Managementcited by 3
  20. 2015When-To-Post on Social Networkscited by 1

Built on by

Cited by year

Citing papers per year: 2023: 12; 2024: 8; 2025: 3.

23 papers.

Sort:

  1. 2023Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlookcited by 109
  2. 2023EasyTPP: Towards Open Benchmarking Temporal Point Processescited by 63
  3. 2023Prompt-augmented Temporal Point Process for Streaming Event Sequencecited by 24
  4. 2024Demonstration of DB-GPT: Next Generation Data Interaction System Empowered by Large Language Modelscited by 9
  5. 2023PILLOW: Enhancing Efficient Instruction Fine-tuning via Prompt Matchingcited by 6
  6. 2025Neural Spatiotemporal Point Processes: Trends and Challengescited by 5
  7. 2023SLOTH: Structured Learning and Task-based Optimization for Time Series Forecasting on Hierarchiescited by 4
  8. 2023Deep Optimal Timing Strategies for Time Seriescited by 2
  9. 2023Self-supervised Adaptive Pre-training of Multilingual Speech Models for Language and Dialect Identificationcited by 2
  10. 2024Adaptive Learning on User Segmentation: Universal to Specific Representation via Bipartite Neural Interactioncited by 1
  11. 2023Enhancing Asynchronous Time Series Forecasting with Contrastive Relational Inferencecited by 0
  12. 2023Enhancing Multilingual Speech Recognition through Language Prompt Tuning and Frame-Level Language Adaptercited by 0
11 more
  1. 2023Amortized Network Intervention to Steer the Excitatory Point Processescited by 0
  2. 2023GMTR: Graph Matching Transformerscited by 0
  3. 2023DarkShot: Lighting Dark Images with Low-Compute and High-Qualitycited by 0
  4. 2024TRLS: A Time Series Representation Learning Framework via Spectrogram for Medical Signal Processingcited by 0
  5. 2024TEN-GUARD: Tensor Decomposition for Backdoor Attack Detection in Deep Neural Networkscited by 0
  6. 2024T-Pixel2Mesh: Combining Global and Local Transformer for 3D Mesh Generation from a Single Imagecited by 0
  7. 2024Strategic Arms with Side Communication Prevail Over Low-Regret MAB Algorithmscited by 0
  8. 2024Towards General Deepfake Detection with Dynamic Curriculumcited by 0
  9. 2024Attr-Int: A Simple and Effective Entity Alignment Framework for Heterogeneous Knowledge Graphscited by 0
  10. 2025Time After Time: Deep-Q Effect Estimation for Interventions on When and What to docited by 0
  11. 2025Mask6D: Masked Pose Priors For 6D Object Pose Estimationcited by 0