# Transformer Hawkes Process

arXiv: https://arxiv.org/abs/2002.09291 · HTML: https://mlsanity.com/paper/2002.09291/

## Key facts

- Cited by 235 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Feb 2020.

## Built on by (most cited first, 30 of 235)

- [CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation](https://mlsanity.com/paper/2107.03502/) (2021)
- [Transformers in Time Series: A Survey](https://mlsanity.com/paper/2202.07125/) (2022)
- [Neural Temporal Point Processes: A Review](https://mlsanity.com/paper/2104.03528/) (2021)
- [Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlook](https://mlsanity.com/paper/2310.10196/) (2023)
- [Transformer Embeddings of Irregularly Spaced Events and Their Participants](https://mlsanity.com/paper/2201.00044/) (2021)
- [EasyTPP: Towards Open Benchmarking Temporal Point Processes](https://mlsanity.com/paper/2307.08097/) (2023)
- [ContiFormer: Continuous-Time Transformer for Irregular Time Series Modeling](https://mlsanity.com/paper/2402.10635/) (2024)
- [A Unified, Scalable Framework for Neural Population Decoding](https://mlsanity.com/paper/2310.16046/) (2023)
- [HYPRO: A Hybridly Normalized Probabilistic Model for Long-Horizon Prediction of Event Sequences](https://mlsanity.com/paper/2210.01753/) (2022)
- [Large Language Models for Forecasting and Anomaly Detection: A Systematic Literature Review](https://mlsanity.com/paper/2402.10350/) (2024)
- [Spatio-temporal Diffusion Point Processes](https://mlsanity.com/paper/2305.12403/) (2023)
- [Event Stream GPT: A Data Pre-processing and Modeling Library for Generative, Pre-trained Transformers over Continuous-time Sequences of Complex Events](https://mlsanity.com/paper/2306.11547/) (2023)
- [Language Models Can Improve Event Prediction by Few-Shot Abductive Reasoning](https://mlsanity.com/paper/2305.16646/) (2023)
- [Neural Point Process for Learning Spatiotemporal Event Dynamics](https://mlsanity.com/paper/2112.06351/) (2021)
- [Anticipatory Music Transformer](https://mlsanity.com/paper/2306.08620/) (2023)
- [Exploring Generative Neural Temporal Point Process](https://mlsanity.com/paper/2208.01874/) (2022)
- [Prompt-augmented Temporal Point Process for Streaming Event Sequence](https://mlsanity.com/paper/2310.04993/) (2023)
- [A survey of dynamic graph neural networks](https://mlsanity.com/paper/2404.18211/) (2024)
- [Bellman Meets Hawkes: Model-Based Reinforcement Learning via Temporal Point Processes](https://mlsanity.com/paper/2201.12569/) (2022)
- [Add and Thin: Diffusion for Temporal Point Processes](https://mlsanity.com/paper/2311.01139/) (2023)
- [Mobility-LLM: Learning Visiting Intentions and Travel Preferences from Human Mobility Data with Large Language Models](https://mlsanity.com/paper/2411.00823/) (2024)
- [Long Horizon Forecasting With Temporal Point Processes](https://mlsanity.com/paper/2101.02815/) (2021)
- [Counterfactual Neural Temporal Point Process for Estimating Causal Influence of Misinformation on Social Media](https://mlsanity.com/paper/2210.07518/) (2022)
- [Learning to Simulate Daily Activities via Modeling Dynamic Human Needs](https://mlsanity.com/paper/2302.10897/) (2023)
- [A Graph Regularized Point Process Model For Event Propagation Sequence](https://mlsanity.com/paper/2211.11758/) (2022)
- [Meta Temporal Point Processes](https://mlsanity.com/paper/2301.12023/) (2023)
- [Decomposable Transformer Point Processes](https://mlsanity.com/paper/2409.18158/) (2024)
- [An Introduction to Deep Survival Analysis Models for Predicting Time-to-Event Outcomes](https://mlsanity.com/paper/2410.01086/) (2024)
- [Mutual Distillation Learning Network for Trajectory-User Linking](https://mlsanity.com/paper/2205.03773/) (2022)
- [Learning Temporal Point Processes for Efficient Retrieval of Continuous Time Event Sequences](https://mlsanity.com/paper/2202.11485/) (2022)

## Abstract (arXiv)

Modern data acquisition routinely produce massive amounts of event sequence data in various domains, such as social media, healthcare, and financial markets. These data often exhibit complicated short-term and long-term temporal dependencies. However, most of the existing recurrent neural network based point process models fail to capture such dependencies, and yield unreliable prediction performance. To address this issue, we propose a Transformer Hawkes Process (THP) model, which leverages the self-attention mechanism to capture long-term dependencies and meanwhile enjoys computational efficiency. Numerical experiments on various datasets show that THP outperforms existing models in terms of both likelihood and event prediction accuracy by a notable margin. Moreover, THP is quite general and can incorporate additional structural knowledge. We provide a concrete example, where THP achieves improved prediction performance for learning multiple point processes when incorporating their relational information.