# Using Sequences of Life-events to Predict Human Lives

arXiv: https://arxiv.org/abs/2306.03009 · HTML: https://mlsanity.com/paper/2306.03009/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 126 references that are in the map.
- First posted on arXiv 5 Jun 2023.

## Builds on (most cited first, 30 of 46)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Rethinking Attention with Performers](https://mlsanity.com/paper/2009.14794/) (2020)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [When Does Label Smoothing Help?](https://mlsanity.com/paper/1906.02629/) (2019)
- [Selective Classification for Deep Neural Networks](https://mlsanity.com/paper/1705.08500/) (2017)
- [A Primer in BERTology: What we know about how BERT works](https://mlsanity.com/paper/2002.12327/) (2020)
- [On the Variance of the Adaptive Learning Rate and Beyond](https://mlsanity.com/paper/1908.03265/) (2019)
- [How to Fine-Tune BERT for Text Classification?](https://mlsanity.com/paper/1905.05583/) (2019)
- [Deep Speech: Scaling up end-to-end speech recognition](https://mlsanity.com/paper/1412.5567/) (2014)
- [ReZero is All You Need: Fast Convergence at Large Depth](https://mlsanity.com/paper/2003.04887/) (2020)
- [BEHRT: Transformer for Electronic Health Records](https://mlsanity.com/paper/1907.09538/) (2019)
- [A Survey on the Explainability of Supervised Machine Learning](https://mlsanity.com/paper/2011.07876/) (2020)
- [Time2Vec: Learning a Vector Representation of Time](https://mlsanity.com/paper/1907.05321/) (2019)
- [All-but-the-Top: Simple and Effective Postprocessing for Word Representations](https://mlsanity.com/paper/1702.01417/) (2017)
- [Music Transformer](https://mlsanity.com/paper/1809.04281/) (2018)
- [Understanding How Dimension Reduction Tools Work: An Empirical Approach to Deciphering t-SNE, UMAP, TriMAP, and PaCMAP for Data Visualization](https://mlsanity.com/paper/2012.04456/) (2020)
- [A Diagnostic Study of Explainability Techniques for Text Classification](https://mlsanity.com/paper/2009.13295/) (2020)

## Built on by (most cited first, 27 of 27)

- [Zero Shot Health Trajectory Prediction Using Transformer](https://mlsanity.com/paper/2407.21124/) (2024)
- [The Benchmarking Epistemology: Validity Theory for Evaluating Machine Learning Models](https://mlsanity.com/paper/2510.23191/) (2025)
- [A Population-to-individual Tuning Framework for Adapting Pretrained LM to On-device User Intent Prediction](https://mlsanity.com/paper/2408.09815/) (2024)
- [Are KANs Effective for Multivariate Time Series Forecasting?](https://mlsanity.com/paper/2408.11306/) (2024)
- [Empathy and the Right to Be an Exception: What LLMs Can and Cannot Do](https://mlsanity.com/paper/2401.14523/) (2024)
- [Estimating Wage Disparities Using Foundation Models](https://mlsanity.com/paper/2409.09894/) (2024)
- [Combining the Strengths of Dutch Survey and Register Data in a Data Challenge to Predict Fertility (PreFer)](https://mlsanity.com/paper/2402.00705/) (2024)
- [Addressing Longstanding Challenges in Cognitive Science with Language Models](https://mlsanity.com/paper/2511.00206/) (2025)
- [Tuning Language Models for Robust Prediction of Diverse User Behaviors](https://mlsanity.com/paper/2505.17682/) (2025)
- [Social Media Use is Predictable from App Sequences: Using LSTM and Transformer Neural Networks to Model Habitual Behavior](https://mlsanity.com/paper/2404.16066/) (2024)
- [Improving Representation Learning of Complex Critical Care Data with ICU-BERT](https://mlsanity.com/paper/2502.19593/) (2025)
- [Explainable embeddings with Distance Explainer](https://mlsanity.com/paper/2505.15516/) (2025)
- [Why can't Epidemiology be automated (yet)?](https://mlsanity.com/paper/2507.15617/) (2025)
- [Using Text-Based Life Trajectories from Swedish Register Data to Predict Residential Mobility with Pretrained Transformers](https://mlsanity.com/paper/2512.07865/) (2025)
- [How Hyper-Datafication Impacts the Sustainability Costs in Frontier AI](https://mlsanity.com/paper/2602.00056/) (2026)
- [On the Unknowable Limits to Prediction](https://mlsanity.com/paper/2411.19223/) (2024)
- [Large language model as user daily behavior data generator: balancing population diversity and individual personality](https://mlsanity.com/paper/2505.17615/) (2025)
- [Scaling Electronic Health Record Foundation Models for Population Health Management](https://mlsanity.com/paper/2506.00209/) (2025)
- [The Book of Life approach: Enabling richness and scale for life course research](https://mlsanity.com/paper/2507.03027/) (2025)
- [Population-Scale Network Embeddings Expose Educational Divides in Network Structure Related to Right-Wing Populist Voting](https://mlsanity.com/paper/2508.21236/) (2025)
- [ASCENDgpt: A Phenotype-Aware Transformer Model for Cardiovascular Risk Prediction from Electronic Health Records](https://mlsanity.com/paper/2509.04485/) (2025)
- [The Intercepted Self: How Generative AI Challenges the Dynamics of the Relational Self](https://mlsanity.com/paper/2509.13391/) (2025)
- [Deep Generative Model for Human Mobility Behavior](https://mlsanity.com/paper/2510.06473/) (2025)
- [LLMs Reading the Rhythms of Daily Life: Aligned Understanding for Behavior Prediction and Generation](https://mlsanity.com/paper/2604.23578/) (2026)
- [SAGA: A Sequence-Adaptive Generative Architecture for Multi-Horizon Probabilistic Forecasting with Adaptive Temporal Conformal Prediction](https://mlsanity.com/paper/2605.19014/) (2026)
- [LifeSentence: Language models can encode human life course trajectories from longitudinal panel data](https://mlsanity.com/paper/2606.11220/) (2026)
- [TEDDY: A Pediatric Foundation Model for Risk Forewarning from ICD-Coded Diagnostic Histories](https://mlsanity.com/paper/2607.14191/) (2026)

## Abstract (arXiv)

Over the past decade, machine learning has revolutionized computers' ability to analyze text through flexible computational models. Due to their structural similarity to written language, transformer-based architectures have also shown promise as tools to make sense of a range of multi-variate sequences from protein-structures, music, electronic health records to weather-forecasts. We can also represent human lives in a way that shares this structural similarity to language. From one perspective, lives are simply sequences of events: People are born, visit the pediatrician, start school, move to a new location, get married, and so on. Here, we exploit this similarity to adapt innovations from natural language processing to examine the evolution and predictability of human lives based on detailed event sequences. We do this by drawing on arguably the most comprehensive registry data in existence, available for an entire nation of more than six million individuals across decades. Our data include information about life-events related to health, education, occupation, income, address, and working hours, recorded with day-to-day resolution. We create embeddings of life-events in a single vector space showing that this embedding space is robust and highly structured. Our models allow us to predict diverse outcomes ranging from early mortality to personality nuances, outperforming state-of-the-art models by a wide margin. Using methods for interpreting deep learning models, we probe the algorithm to understand the factors that enable our predictions. Our framework allows researchers to identify new potential mechanisms that impact life outcomes and associated possibilities for personalized interventions.