# HYPRO: A Hybridly Normalized Probabilistic Model for Long-Horizon Prediction of Event Sequences

arXiv: https://arxiv.org/abs/2210.01753 · HTML: https://mlsanity.com/paper/2210.01753/

## Key facts

- Cited by 57 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 57 references that are in the map.
- First posted on arXiv 4 Oct 2022.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [Sequence Level Training with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06732/) (2015)
- [Transformer Hawkes Process](https://mlsanity.com/paper/2002.09291/) (2020)
- [Intensity-Free Learning of Temporal Point Processes](https://mlsanity.com/paper/1909.12127/) (2019)
- [On the Turing Completeness of Modern Neural Network Architectures](https://mlsanity.com/paper/1901.03429/) (2019)
- [Fully Neural Network based Model for General Temporal Point Processes](https://mlsanity.com/paper/1905.09690/) (2019)
- [Noise Contrastive Estimation and Negative Sampling for Conditional Models: Consistency and Statistical Efficiency](https://mlsanity.com/paper/1809.01812/) (2018)
- [Long Text Generation via Adversarial Training with Leaked Information](https://mlsanity.com/paper/1709.08624/) (2017)
- [Real or Fake? Learning to Discriminate Machine from Human Generated Text](https://mlsanity.com/paper/1906.03351/) (2019)
- [Residual Energy-Based Models for Text Generation](https://mlsanity.com/paper/2004.11714/) (2020)
- [Modeling The Intensity Function Of Point Process Via Recurrent Neural Networks](https://mlsanity.com/paper/1705.08982/) (2017)
- [Transformer Embeddings of Irregularly Spaced Events and Their Participants](https://mlsanity.com/paper/2201.00044/) (2021)
- [Shape and Time Distortion Loss for Training Deep Time Series Forecasting Models](https://mlsanity.com/paper/1909.09020/) (2019)
- [Trajectory Prediction with Latent Belief Energy-Based Model](https://mlsanity.com/paper/2104.03086/) (2021)
- [Limitations of Autoregressive Models and Their Alternatives](https://mlsanity.com/paper/2010.11939/) (2020)
- [What Context Features Can Transformer Language Models Use?](https://mlsanity.com/paper/2106.08367/) (2021)
- [Identifying Coordinated Accounts on Social Media through Hidden Influence and Group Behaviours](https://mlsanity.com/paper/2008.11308/) (2020)
- [Long Text Generation by Modeling Sentence-Level and Discourse-Level Coherence](https://mlsanity.com/paper/2105.08963/) (2021)
- [Imputing Missing Events in Continuous-Time Event Streams](https://mlsanity.com/paper/1905.05570/) (2019)
- [Neural Temporal Point Processes For Modelling Electronic Health Records](https://mlsanity.com/paper/2007.13794/) (2020)
- [Learning Energy-based Spatial-Temporal Generative ConvNets for Dynamic Patterns](https://mlsanity.com/paper/1909.11975/) (2019)
- [Noise-Contrastive Estimation for Multivariate Point Processes](https://mlsanity.com/paper/2011.00717/) (2020)
- [Long Horizon Forecasting With Temporal Point Processes](https://mlsanity.com/paper/2101.02815/) (2021)
- [User-Dependent Neural Sequence Models for Continuous-Time Event Data](https://mlsanity.com/paper/2011.03231/) (2020)
- [Neural Datalog Through Time: Informed Temporal Modeling via Logical Specification](https://mlsanity.com/paper/2006.16723/) (2020)
- [Joint Modeling of Event Sequence and Time Series with Attentional Twin Recurrent Neural Networks](https://mlsanity.com/paper/1703.08524/) (2017)

## Built on by (most cited first, 30 of 57)

- [Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlook](https://mlsanity.com/paper/2310.10196/) (2023)
- [EasyTPP: Towards Open Benchmarking Temporal Point Processes](https://mlsanity.com/paper/2307.08097/) (2023)
- [Empowering Time Series Analysis with Large Language Models: A Survey](https://mlsanity.com/paper/2402.03182/) (2024)
- [Language Models Can Improve Event Prediction by Few-Shot Abductive Reasoning](https://mlsanity.com/paper/2305.16646/) (2023)
- [Prompt-augmented Temporal Point Process for Streaming Event Sequence](https://mlsanity.com/paper/2310.04993/) (2023)
- [Decomposable Transformer Point Processes](https://mlsanity.com/paper/2409.18158/) (2024)
- [HoTPP Benchmark: Are We Good at the Long Horizon Events Forecasting?](https://mlsanity.com/paper/2406.14341/) (2024)
- [Deep Continuous-Time State-Space Models for Marked Event Sequences](https://mlsanity.com/paper/2412.19634/) (2024)
- [Interacting Diffusion Processes for Event Sequence Forecasting](https://mlsanity.com/paper/2310.17800/) (2023)
- [In-Context Learning of Temporal Point Processes with Foundation Inference Models](https://mlsanity.com/paper/2509.24762/) (2025)
- [Demonstration of DB-GPT: Next Generation Data Interaction System Empowered by Large Language Models](https://mlsanity.com/paper/2404.10209/) (2024)
- [EventFlow: Forecasting Temporal Point Processes with Flow Matching](https://mlsanity.com/paper/2410.07430/) (2024)
- [Non-Autoregressive Diffusion-based Temporal Point Processes for Continuous-Time Long-Term Event Prediction](https://mlsanity.com/paper/2311.01033/) (2023)
- [Detecting the Future: All-at-Once Event Sequence Forecasting with Horizon Matching](https://mlsanity.com/paper/2408.13131/) (2024)
- [Byte-token Enhanced Language Models for Temporal Point Processes Analysis](https://mlsanity.com/paper/2502.07139/) (2025)
- [SLOTH: Structured Learning and Task-based Optimization for Time Series Forecasting on Hierarchies](https://mlsanity.com/paper/2302.05650/) (2023)
- [Is Score Matching Suitable for Estimating Point Processes?](https://mlsanity.com/paper/2410.04037/) (2024)
- [LAST SToP For Modeling Asynchronous Time Series](https://mlsanity.com/paper/2502.01922/) (2025)
- [PEER: Expertizing Domain-Specific Tasks with a Multi-Agent Framework and Tuning Methods](https://mlsanity.com/paper/2407.06985/) (2024)
- [ROMAS: A Role-Based Multi-Agent System for Database monitoring and Planning](https://mlsanity.com/paper/2412.13520/) (2024)
- [Advances in Temporal Point Processes: Bayesian, Neural, and LLM Approaches](https://mlsanity.com/paper/2501.14291/) (2025)
- [TPP-SD: Accelerating Transformer Point Process Sampling with Speculative Decoding](https://mlsanity.com/paper/2507.09252/) (2025)
- [Deep Optimal Timing Strategies for Time Series](https://mlsanity.com/paper/2310.05479/) (2023)
- [Self-supervised Adaptive Pre-training of Multilingual Speech Models for Language and Dialect Identification](https://mlsanity.com/paper/2312.07338/) (2023)
- [Probabilistic Modeling for Sequences of Sets in Continuous-Time](https://mlsanity.com/paper/2312.15045/) (2023)
- [ADiff4TPP: Asynchronous Diffusion Models for Temporal Point Processes](https://mlsanity.com/paper/2504.20411/) (2025)
- [DORAEMON: A Unified Library for Visual Object Modeling and Representation Learning at Scale](https://mlsanity.com/paper/2511.04394/) (2025)
- [Adaptive Learning on User Segmentation: Universal to Specific Representation via Bipartite Neural Interaction](https://mlsanity.com/paper/2409.14945/) (2024)
- [A Bayesian Mixture Model of Temporal Point Processes with Determinantal Point Process Prior](https://mlsanity.com/paper/2411.04397/) (2024)
- [Still Competitive: Revisiting Recurrent Models for Irregular Time Series Prediction](https://mlsanity.com/paper/2510.16161/) (2025)

## Abstract (arXiv)

In this paper, we tackle the important yet under-investigated problem of making long-horizon prediction of event sequences. Existing state-of-the-art models do not perform well at this task due to their autoregressive structure. We propose HYPRO, a hybridly normalized probabilistic model that naturally fits this task: its first part is an autoregressive base model that learns to propose predictions; its second part is an energy function that learns to reweight the proposals such that more realistic predictions end up with higher probabilities. We also propose efficient training and inference algorithms for this model. Experiments on multiple real-world datasets demonstrate that our proposed HYPRO model can significantly outperform previous models at making long-horizon predictions of future events. We also conduct a range of ablation studies to investigate the effectiveness of each component of our proposed methods.