# Byte-token Enhanced Language Models for Temporal Point Processes Analysis

arXiv: https://arxiv.org/abs/2502.07139 · HTML: https://mlsanity.com/paper/2502.07139/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 49 references that are in the map.
- First posted on arXiv 11 Feb 2025.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [MiniCPM-V: A GPT-4V Level MLLM on Your Phone](https://mlsanity.com/paper/2408.01800/) (2024)
- [Extending Context Window of Large Language Models via Positional Interpolation](https://mlsanity.com/paper/2306.15595/) (2023)
- [ChemCrow: Augmenting large-language models with chemistry tools](https://mlsanity.com/paper/2304.05376/) (2023)
- [ByT5: Towards a token-free future with pre-trained byte-to-byte models](https://mlsanity.com/paper/2105.13626/) (2021)
- [Any-to-Any Generation via Composable Diffusion](https://mlsanity.com/paper/2305.11846/) (2023)
- [One Transformer Fits All Distributions in Multi-Modal Diffusion at Scale](https://mlsanity.com/paper/2303.06555/) (2023)
- [Byte Latent Transformer: Patches Scale Better Than Tokens](https://mlsanity.com/paper/2412.09871/) (2024)
- [Intensity-Free Learning of Temporal Point Processes](https://mlsanity.com/paper/1909.12127/) (2019)
- [Modeling The Intensity Function Of Point Process Via Recurrent Neural Networks](https://mlsanity.com/paper/1705.08982/) (2017)
- [Transformer Embeddings of Irregularly Spaced Events and Their Participants](https://mlsanity.com/paper/2201.00044/) (2021)
- [HYPRO: A Hybridly Normalized Probabilistic Model for Long-Horizon Prediction of Event Sequences](https://mlsanity.com/paper/2210.01753/) (2022)
- [Language Models Can Improve Event Prediction by Few-Shot Abductive Reasoning](https://mlsanity.com/paper/2305.16646/) (2023)
- [Lecture Notes: Temporal Point Processes and the Conditional Intensity Function](https://mlsanity.com/paper/1806.00221/) (2018)
- [COEVOLVE: A Joint Point Process Model for Information Diffusion and Network Co-evolution](https://mlsanity.com/paper/1507.02293/) (2015)
- [Prompt-augmented Temporal Point Process for Streaming Event Sequence](https://mlsanity.com/paper/2310.04993/) (2023)
- [Feature Driven and Point Process Approaches for Popularity Prediction](https://mlsanity.com/paper/1608.04862/) (2016)
- [Interval-censored Transformer Hawkes: Detecting Information Operations using the Reaction of Social Systems](https://mlsanity.com/paper/2211.14114/) (2022)
- [Interpretable Transformer Hawkes Processes: Unveiling Complex Interactions in Social Networks](https://mlsanity.com/paper/2405.16059/) (2024)
- [Modeling Popularity in Asynchronous Social Media Streams with Recurrent Neural Networks](https://mlsanity.com/paper/1804.02101/) (2018)

## Built on by (most cited first, 6 of 6)

- [DanmakuTPPBench: A Multi-modal Benchmark for Temporal Point Process Modeling and Understanding](https://mlsanity.com/paper/2505.18411/) (2025)
- [MILM: Large Language Models for Multimodal Irregular Time Series with Informative Sampling](https://mlsanity.com/paper/2605.13711/) (2026)
- [Temporal Tokenization Strategies for Event Sequence Modeling with Large Language Models](https://mlsanity.com/paper/2512.13618/) (2025)
- [Long-range Modeling and Processing of Multimodal Event Sequences](https://mlsanity.com/paper/2602.01125/) (2026)
- [Temporal Flattening in LLM-Generated Text: Comparing Human and LLM Writing Trajectories](https://mlsanity.com/paper/2604.12097/) (2026)
- [Can LLMs Access their World Knowledge for Event Prediction?](https://mlsanity.com/paper/2610.19021/) (2026)

## Abstract (arXiv)

Temporal Point Processes (TPPs) have been widely used for modeling event sequences on the Web, such as user reviews, social media posts, and online transactions. However, traditional TPP models often struggle to effectively incorporate the rich textual descriptions that accompany these events, while Large Language Models (LLMs), despite their remarkable text processing capabilities, lack mechanisms for handling the temporal dynamics inherent in Web-based event sequences. To bridge this gap, we introduce Language-TPP, a unified framework that seamlessly integrates TPPs with LLMs for enhanced Web event sequence modeling. Our key innovation is a novel temporal encoding mechanism that converts continuous time intervals into specialized byte-tokens, enabling direct integration with standard language model architectures for TPP modeling without requiring TPP-specific modifications. This approach allows Language-TPP to achieve state-of-the-art performance across multiple TPP benchmarks, including event time prediction and type prediction, on real-world Web datasets spanning e-commerce reviews, social media and online Q&A platforms. More importantly, we demonstrate that our unified framework unlocks new capabilities for TPP research: incorporating temporal information improves the quality of generated event descriptions, as evidenced by enhanced ROUGE-L scores, and better aligned sentiment distributions. Through comprehensive experiments, including qualitative analysis of learned distributions and scalability evaluations on long sequences, we show that Language-TPP effectively captures both temporal dynamics and textual patterns in Web user behavior, with important implications for content generation, user behavior understanding, and Web platform applications. Code is available at https://github.com/qykong/Language-TPP.