# Universal Transformer Hawkes Process with Adaptive Recursive Iteration

arXiv: https://arxiv.org/abs/2112.14479 · HTML: https://mlsanity.com/paper/2112.14479/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 37 references that are in the map.
- First posted on arXiv 29 Dec 2021.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Universal Transformers](https://mlsanity.com/paper/1807.03819/) (2018)
- [Adaptive Computation Time for Recurrent Neural Networks](https://mlsanity.com/paper/1603.08983/) (2016)
- [Transformer Hawkes Process](https://mlsanity.com/paper/2002.09291/) (2020)
- [Supervised Reinforcement Learning with Recurrent Neural Network for Dynamic Treatment Recommendation](https://mlsanity.com/paper/1807.01473/) (2018)
- [Modeling The Intensity Function Of Point Process Via Recurrent Neural Networks](https://mlsanity.com/paper/1705.08982/) (2017)
- [SEISMIC: A Self-Exciting Point Process Model for Predicting Tweet Popularity](https://mlsanity.com/paper/1506.02594/) (2015)
- [Learning Granger Causality for Hawkes Processes](https://mlsanity.com/paper/1602.04511/) (2016)
- [Language Models with Transformers](https://mlsanity.com/paper/1904.09408/) (2019)
- [TiDeH: Time-Dependent Hawkes Process for Predicting Retweet Dynamics](https://mlsanity.com/paper/1603.09449/) (2016)

## Built on by (most cited first, 3 of 3)

- [Mamba Hawkes Process](https://mlsanity.com/paper/2407.05302/) (2024)
- [RoTHP: Rotary Position Embedding-based Transformer Hawkes Process](https://mlsanity.com/paper/2405.06985/) (2024)
- [L-FNO: Lorentzian Fourier Neural Operator for Stochastic Event Dynamics](https://mlsanity.com/paper/2608.13562/) (2026)

## Abstract (arXiv)

Asynchronous events sequences are widely distributed in the natural world and human activities, such as earthquakes records, users activities in social media and so on. How to distill the information from these seemingly disorganized data is a persistent topic that researchers focus on. The one of the most useful model is the point process model, and on the basis, the researchers obtain many noticeable results. Moreover, in recent years, point process models on the foundation of neural networks, especially recurrent neural networks (RNN) are proposed and compare with the traditional models, their performance are greatly improved. Enlighten by transformer model, which can learning sequence data efficiently without recurrent and convolutional structure, transformer Hawkes process is come out, and achieves state-of-the-art performance. However, there is some research proving that the re-introduction of recursive calculations in transformer can further improve transformers performance. Thus, we come out with a new kind of transformer Hawkes process model, universal transformer Hawkes process (UTHP), which contains both recursive mechanism and self-attention mechanism, and to improve the local perception ability of the model, we also introduce convolutional neural network (CNN) in the position-wise-feed-forward part. We conduct experiments on several datasets to validate the effectiveness of UTHP and explore the changes after the introduction of the recursive mechanism. These experiments on multiple datasets demonstrate that the performance of our proposed new model has a certain improvement compared with the previous state-of-the-art models.