# ProActive: Self-Attentive Temporal Point Process Flows for Activity Sequences arXiv: https://arxiv.org/abs/2206.05291 ยท HTML: https://mlsanity.com/paper/2206.05291/ ## Key facts - Cited by 11 papers in the map (data through 14 Oct 2026). - Builds on 30 of its 51 references that are in the map. - First posted on arXiv 10 Jun 2022. ## Builds on (most cited first, 30 of 30) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014) - [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016) - [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022) - [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015) - [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018) - [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018) - [Improving Variational Inference with Inverse Autoregressive Flow](https://mlsanity.com/paper/1606.04934/) (2016) - [Transformer Hawkes Process](https://mlsanity.com/paper/2002.09291/) (2020) - [Intensity-Free Learning of Temporal Point Processes](https://mlsanity.com/paper/1909.12127/) (2019) - [Fully Neural Network based Model for General Temporal Point Processes](https://mlsanity.com/paper/1905.09690/) (2019) - [Neural Temporal Point Processes: A Review](https://mlsanity.com/paper/2104.03528/) (2021) - [When will you do what? - Anticipating Temporal Occurrences of Activities](https://mlsanity.com/paper/1804.00892/) (2018) - [Every Moment Counts: Dense Detailed Labeling of Actions in Complex Videos](https://mlsanity.com/paper/1507.05738/) (2015) - [Modeling The Intensity Function Of Point Process Via Recurrent Neural Networks](https://mlsanity.com/paper/1705.08982/) (2017) - [SEISMIC: A Self-Exciting Point Process Model for Predicting Tweet Popularity](https://mlsanity.com/paper/1506.02594/) (2015) - [Identifying Coordinated Accounts on Social Media through Hidden Influence and Group Behaviours](https://mlsanity.com/paper/2008.11308/) (2020) - [Wasserstein Learning of Deep Generative Point Process Models](https://mlsanity.com/paper/1705.08051/) (2017) - [Rethinking the Discount Factor in Reinforcement Learning: A Decision Theoretic Approach](https://mlsanity.com/paper/1902.02893/) (2019) - [A Variational Auto-Encoder Model for Stochastic Point Processes](https://mlsanity.com/paper/1904.03273/) (2019) - [Expecting to be HIP: Hawkes Intensity Processes for Social Media Popularity](https://mlsanity.com/paper/1602.06033/) (2016) - [Self-Attentive Hawkes Processes](https://mlsanity.com/paper/1907.07561/) (2019) - [SenseGen: A Deep Learning Architecture for Synthetic Sensor Data Generation](https://mlsanity.com/paper/1701.08886/) (2017) - [Long Horizon Forecasting With Temporal Point Processes](https://mlsanity.com/paper/2101.02815/) (2021) - [Learning Temporal Point Processes for Efficient Retrieval of Continuous Time Event Sequences](https://mlsanity.com/paper/2202.11485/) (2022) - [Stimuli-Sensitive Hawkes Processes for Personalized Student Procrastination Modeling](https://mlsanity.com/paper/2102.00089/) (2021) - [Region Invariant Normalizing Flows for Mobility Transfer](https://mlsanity.com/paper/2109.05738/) (2021) - [Doing More with Less: Overcoming Data Scarcity for POI Recommendation via Cross-Region Transfer](https://mlsanity.com/paper/2201.06095/) (2022) - [Point Process Flows](https://mlsanity.com/paper/1910.08281/) (2019) - [Learning Person Trajectory Representations for Team Activity Analysis](https://mlsanity.com/paper/1706.00893/) (2017) ## Built on by (most cited first, 11 of 11) - [ContiFormer: Continuous-Time Transformer for Irregular Time Series Modeling](https://mlsanity.com/paper/2402.10635/) (2024) - [Learning to Simulate Daily Activities via Modeling Dynamic Human Needs](https://mlsanity.com/paper/2302.10897/) (2023) - [A Survey on Deep Learning Techniques for Action Anticipation](https://mlsanity.com/paper/2309.17257/) (2023) - [Modeling Events and Interactions through Temporal Processes -- A Survey](https://mlsanity.com/paper/2303.06067/) (2023) - [Modeling Continuous Time Sequences with Intermittent Observations using Marked Temporal Point Processes](https://mlsanity.com/paper/2206.12414/) (2022) - [Modeling Spatial Trajectories using Coarse-Grained Smartphone Logs](https://mlsanity.com/paper/2208.13775/) (2022) - [PROPER Agents: Proactivity Driven Personalized Agents for Advancing Knowledge Gap Navigation](https://mlsanity.com/paper/2601.09926/) (2026) - [Retrieving Continuous Time Event Sequences using Neural Temporal Point Processes with Learnable Hashing](https://mlsanity.com/paper/2307.09613/) (2023) - [Tapestry of Time and Actions: Modeling Human Activity Sequences using Temporal Point Process Flows](https://mlsanity.com/paper/2307.10305/) (2023) - [Self-supervised New Activity Detection in Sensor-based Smart Environments](https://mlsanity.com/paper/2401.10288/) (2024) - [Transfer Learning with Point Transformers](https://mlsanity.com/paper/2404.00846/) (2024) ## Abstract (arXiv) Any human activity can be represented as a temporal sequence of actions performed to achieve a certain goal. Unlike machine-made time series, these action sequences are highly disparate as the time taken to finish a similar action might vary between different persons. Therefore, understanding the dynamics of these sequences is essential for many downstream tasks such as activity length prediction, goal prediction, etc. Existing neural approaches that model an activity sequence are either limited to visual data or are task specific, i.e., limited to next action or goal prediction. In this paper, we present ProActive, a neural marked temporal point process (MTPP) framework for modeling the continuous-time distribution of actions in an activity sequence while simultaneously addressing three high-impact problems -- next action prediction, sequence-goal prediction, and end-to-end sequence generation. Specifically, we utilize a self-attention module with temporal normalizing flows to model the influence and the inter-arrival times between actions in a sequence. Moreover, for time-sensitive prediction, we perform an early detection of sequence goal via a constrained margin-based optimization procedure. This in-turn allows ProActive to predict the sequence goal using a limited number of actions. Extensive experiments on sequences derived from three activity recognition datasets show the significant accuracy boost of ProActive over the state-of-the-art in terms of action and goal prediction, and the first-ever application of end-to-end action sequence generation.