# Temporal Tokenization Strategies for Event Sequence Modeling with Large Language Models

arXiv: https://arxiv.org/abs/2512.13618 · HTML: https://mlsanity.com/paper/2512.13618/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 25 references that are in the map.
- First posted on arXiv 15 Dec 2025.

## Builds on (most cited first, 10 of 10)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Neural Temporal Point Processes: A Review](https://mlsanity.com/paper/2104.03528/) (2021)
- [A Review of Self-Exciting Spatio-Temporal Point Processes and Their Applications](https://mlsanity.com/paper/1708.02647/) (2017)
- [DanmakuTPPBench: A Multi-modal Benchmark for Temporal Point Process Modeling and Understanding](https://mlsanity.com/paper/2505.18411/) (2025)
- [Retrieval of Temporal Event Sequences from Textual Descriptions](https://mlsanity.com/paper/2410.14043/) (2024)
- [Byte-token Enhanced Language Models for Temporal Point Processes Analysis](https://mlsanity.com/paper/2502.07139/) (2025)
- [Date Fragments: A Hidden Bottleneck of Tokenization for Temporal Reasoning](https://mlsanity.com/paper/2505.16088/) (2025)
- [RHYTHM: Reasoning with Hierarchical Temporal Tokenization for Human Mobility](https://mlsanity.com/paper/2509.23115/) (2025)

## Built on by (most cited first, 1 of 1)

- [AutoMARS: Automatic and Scalable Multi-Agent Large Language Model Framework for Supply Chain Risk Assessment](https://mlsanity.com/paper/2610.17206/) (2026)

## Abstract (arXiv)

Representing continuous time is a critical and under-explored challenge in modeling temporal event sequences with large language models (LLMs). Various strategies like byte-level representations or calendar tokens have been proposed. However, the optimal approach remains unclear, especially given the diverse statistical distributions of real-world event data, which range from smooth log-normal to discrete, spiky patterns. This paper presents a systematic empirical study of temporal tokenization for modeling event sequences with LLMs, comparing distinct encoding strategies: naive numeric strings, high-precision byte-level representations, human-semantic calendar tokens, classic uniform binning, and adaptive residual scalar quantization. We evaluate these strategies by fine-tuning LLMs on real-world datasets that exemplify these diverse distributions. Our analysis reveals that no single strategy is universally superior; instead, prediction performance depends heavily on aligning the tokenizer with the data's statistical properties, highlighting temporal tokenization as a critical yet often overlooked design dimension in LLM-based event modeling.