Using Sequences of Life-events to Predict Human Lives
Not on the map. Why?
Abstract (arXiv)
Over the past decade, machine learning has revolutionized computers' ability to analyze text through flexible computational models. Due to their structural similarity to written language, transformer-based architectures have also shown promise as tools to make sense of a range of multi-variate sequences from protein-structures, music, electronic health records to weather-forecasts. We can also represent human lives in a way that shares this structural similarity to language. From one perspective, lives are simply sequences of events: People are born, visit the pediatrician, start school, move to a new location, get married, and so on. Here, we exploit this similarity to adapt innovations from natural language processing to examine the evolution and predictability of human lives based on detailed event sequences. We do this by drawing on arguably the most comprehensive registry data in existence, available for an entire nation of more than six million individuals across decades. Our data include information about life-events related to health, education, occupation, income, address, and working hours, recorded with day-to-day resolution. We create embeddings of life-events in a single vector space showing that this embedding space is robust and highly structured. Our models allow us to predict diverse outcomes ranging from early mortality to personality nuances, outperforming state-of-the-art models by a wide margin. Using methods for interpreting deep learning models, we probe the algorithm to understand the factors that enable our predictions. Our framework allows researchers to identify new potential mechanisms that impact life outcomes and associated possibilities for personalized interventions.
Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). Builds on 46 of its 126 references that are in the map. First posted on arXiv 5 Jun 2023.
Builds on
46 of 126 references.
References · 46
- 2017Attention Is All You Needcited by 60,361
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
- 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
- 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
- 2020Longformer: The Long-Document Transformercited by 3,413
- 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
- 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
- 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
34 more
- 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,367
- 2019A Survey on Bias and Fairness in Machine Learningcited by 1,917
- 2020Rethinking Attention with Performerscited by 1,601
- 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
- 2019When Does Label Smoothing Help?cited by 988
- 2017Selective Classification for Deep Neural Networkscited by 852
- 2020A Primer in BERTology: What we know about how BERT workscited by 815
- 2019On the Variance of the Adaptive Learning Rate and Beyondcited by 772
- 2019How to Fine-Tune BERT for Text Classification?cited by 411
- 2014Deep Speech: Scaling up end-to-end speech recognitioncited by 354
- 2020ReZero is All You Need: Fast Convergence at Large Depthcited by 264
- 2019BEHRT: Transformer for Electronic Health Recordscited by 263
- 2020A Survey on the Explainability of Supervised Machine Learningcited by 227
- 2019Time2Vec: Learning a Vector Representation of Timecited by 213
- 2017All-but-the-Top: Simple and Effective Postprocessing for Word Representationscited by 212
- 2018Music Transformercited by 211
- 2020Understanding How Dimension Reduction Tools Work: An Empirical Approach to Deciphering t-SNE, UMAP, TriMAP, and PaCMAP for Data Visualizationcited by 173
- 2020A Diagnostic Study of Explainability Techniques for Text Classificationcited by 159
- 2019Transformers without Tears: Improving the Normalization of Self-Attentioncited by 153
- 2020The elephant in the interpretability room: Why use attention as explanation when we have saliency methods?cited by 122
- 2022Should You Mask 15% in Masked Language Modeling?cited by 118
- 2020End-to-end Lane Shape Prediction with Transformerscited by 111
- 2018The Geometry of Culture: Analyzing Meaning through Word Embeddingscited by 87
- 2018From Word to Sense Embeddings: A Survey on Vector Representations of Meaningcited by 49
- 2020On Interpretability of Deep Learning based Skin Lesion Classifiers using Concept Activation Vectorscited by 39
- 2020Multi-Loss Weighting with Coefficient of Variationscited by 36
- 2019Saliency-driven Word Alignment Interpretation for Neural Machine Translationcited by 27
- 2021Learning to Remove: Towards Isotropic Pre-trained BERT Embeddingcited by 23
- 2018Sigsoftmax: Reanalysis of the Softmax Bottleneckcited by 20
- 2021MELONS: generating melody with long-term structure using transformers and structure graphcited by 18
- 2022Class Distance Weighted Cross-Entropy Loss for Ulcerative Colitis Severity Estimationcited by 17
- 2021Towards physically consistent data-driven weather forecasting: Integrating data assimilation with equivariance-preserving deep spatial transformerscited by 15
- 2018Beyond Weight Tying: Learning Joint Input-Output Embeddings for Neural Machine Translationcited by 5
- 2020Improving Positive Unlabeled Learning: Practical AUL Estimation and New Training Method for Extremely Imbalanced Data Setscited by 3
Built on by
Citing papers per year: 2024: 8; 2025: 14; 2026: 5.
27 papers.
Sort:
- 2024Zero Shot Health Trajectory Prediction Using Transformercited by 77
- 2025The Benchmarking Epistemology: Validity Theory for Evaluating Machine Learning Modelscited by 13
- 2024A Population-to-individual Tuning Framework for Adapting Pretrained LM to On-device User Intent Predictioncited by 6
- 2024Are KANs Effective for Multivariate Time Series Forecasting?cited by 6
- 2024Empathy and the Right to Be an Exception: What LLMs Can and Cannot Docited by 4
- 2024Estimating Wage Disparities Using Foundation Modelscited by 4
- 2024Combining the Strengths of Dutch Survey and Register Data in a Data Challenge to Predict Fertility (PreFer)cited by 3
- 2025Addressing Longstanding Challenges in Cognitive Science with Language Modelscited by 3
- 2025Tuning Language Models for Robust Prediction of Diverse User Behaviorscited by 2
- 2024Social Media Use is Predictable from App Sequences: Using LSTM and Transformer Neural Networks to Model Habitual Behaviorcited by 1
- 2025Improving Representation Learning of Complex Critical Care Data with ICU-BERTcited by 1
- 2025Explainable embeddings with Distance Explainercited by 1
15 more
- 2025Why can't Epidemiology be automated (yet)?cited by 1
- 2025Using Text-Based Life Trajectories from Swedish Register Data to Predict Residential Mobility with Pretrained Transformerscited by 1
- 2026How Hyper-Datafication Impacts the Sustainability Costs in Frontier AIcited by 1
- 2024On the Unknowable Limits to Predictioncited by 0
- 2025Large language model as user daily behavior data generator: balancing population diversity and individual personalitycited by 0
- 2025Scaling Electronic Health Record Foundation Models for Population Health Managementcited by 0
- 2025The Book of Life approach: Enabling richness and scale for life course researchcited by 0
- 2025Population-Scale Network Embeddings Expose Educational Divides in Network Structure Related to Right-Wing Populist Votingcited by 0
- 2025ASCENDgpt: A Phenotype-Aware Transformer Model for Cardiovascular Risk Prediction from Electronic Health Recordscited by 0
- 2025The Intercepted Self: How Generative AI Challenges the Dynamics of the Relational Selfcited by 0
- 2025Deep Generative Model for Human Mobility Behaviorcited by 0
- 2026LLMs Reading the Rhythms of Daily Life: Aligned Understanding for Behavior Prediction and Generationcited by 0
- 2026SAGA: A Sequence-Adaptive Generative Architecture for Multi-Horizon Probabilistic Forecasting with Adaptive Temporal Conformal Predictioncited by 0
- 2026LifeSentence: Language models can encode human life course trajectories from longitudinal panel datacited by 0
- 2026TEDDY: A Pediatric Foundation Model for Risk Forewarning from ICD-Coded Diagnostic Historiescited by 0
- 2026TEDDY: A Pediatric Foundation Model for Risk Forewarning from ICD-Coded Diagnostic Histories15 Jul 2026
- 2026SAGA: A Sequence-Adaptive Generative Architecture for Multi-Horizon Probabilistic Forecasting with Adaptive Temporal Conformal Prediction18 May 2026
- 2026LifeSentence: Language models can encode human life course trajectories from longitudinal panel data14 May 2026
- 2026LLMs Reading the Rhythms of Daily Life: Aligned Understanding for Behavior Prediction and Generation26 Apr 2026
- 2026How Hyper-Datafication Impacts the Sustainability Costs in Frontier AI20 Jan 2026
- 2025Using Text-Based Life Trajectories from Swedish Register Data to Predict Residential Mobility with Pretrained Transformers26 Nov 2025
- 2025Addressing Longstanding Challenges in Cognitive Science with Language Models31 Oct 2025
- 2025The Benchmarking Epistemology: Validity Theory for Evaluating Machine Learning Models27 Oct 2025
- 2025Deep Generative Model for Human Mobility Behavior7 Oct 2025
- 2025The Intercepted Self: How Generative AI Challenges the Dynamics of the Relational Self16 Sep 2025
- 2025ASCENDgpt: A Phenotype-Aware Transformer Model for Cardiovascular Risk Prediction from Electronic Health Records31 Aug 2025
- 2025Population-Scale Network Embeddings Expose Educational Divides in Network Structure Related to Right-Wing Populist Voting28 Aug 2025
15 more
- 2025Why can't Epidemiology be automated (yet)?21 Jul 2025
- 2025The Book of Life approach: Enabling richness and scale for life course research2 Jul 2025
- 2025Scaling Electronic Health Record Foundation Models for Population Health Management30 May 2025
- 2025Tuning Language Models for Robust Prediction of Diverse User Behaviors23 May 2025
- 2025Large language model as user daily behavior data generator: balancing population diversity and individual personality23 May 2025
- 2025Explainable embeddings with Distance Explainer21 May 2025
- 2025Improving Representation Learning of Complex Critical Care Data with ICU-BERT26 Feb 2025
- 2024On the Unknowable Limits to Prediction28 Nov 2024
- 2024Estimating Wage Disparities Using Foundation Models15 Sep 2024
- 2024Are KANs Effective for Multivariate Time Series Forecasting?21 Aug 2024
- 2024A Population-to-individual Tuning Framework for Adapting Pretrained LM to On-device User Intent Prediction19 Aug 2024
- 2024Zero Shot Health Trajectory Prediction Using Transformer30 Jul 2024
- 2024Social Media Use is Predictable from App Sequences: Using LSTM and Transformer Neural Networks to Model Habitual Behavior20 Apr 2024
- 2024Combining the Strengths of Dutch Survey and Register Data in a Data Challenge to Predict Fertility (PreFer)1 Feb 2024
- 2024Empathy and the Right to Be an Exception: What LLMs Can and Cannot Do25 Jan 2024
- 2024Zero Shot Health Trajectory Prediction Using Transformercited by 77
- 2025The Benchmarking Epistemology: Validity Theory for Evaluating Machine Learning Modelscited by 13
- 2024A Population-to-individual Tuning Framework for Adapting Pretrained LM to On-device User Intent Predictioncited by 6
- 2024Are KANs Effective for Multivariate Time Series Forecasting?cited by 6
- 2024Empathy and the Right to Be an Exception: What LLMs Can and Cannot Docited by 4
- 2024Estimating Wage Disparities Using Foundation Modelscited by 4
- 2024Combining the Strengths of Dutch Survey and Register Data in a Data Challenge to Predict Fertility (PreFer)cited by 3
- 2025Addressing Longstanding Challenges in Cognitive Science with Language Modelscited by 3
- 2025Tuning Language Models for Robust Prediction of Diverse User Behaviorscited by 2
- 2024Social Media Use is Predictable from App Sequences: Using LSTM and Transformer Neural Networks to Model Habitual Behaviorcited by 1
- 2025Improving Representation Learning of Complex Critical Care Data with ICU-BERTcited by 1
- 2025Explainable embeddings with Distance Explainercited by 1
12 most cited of 27 All 27 (JSON)
- 2026TEDDY: A Pediatric Foundation Model for Risk Forewarning from ICD-Coded Diagnostic Histories15 Jul 2026
- 2026SAGA: A Sequence-Adaptive Generative Architecture for Multi-Horizon Probabilistic Forecasting with Adaptive Temporal Conformal Prediction18 May 2026
- 2026LifeSentence: Language models can encode human life course trajectories from longitudinal panel data14 May 2026
- 2026LLMs Reading the Rhythms of Daily Life: Aligned Understanding for Behavior Prediction and Generation26 Apr 2026
- 2026How Hyper-Datafication Impacts the Sustainability Costs in Frontier AI20 Jan 2026
- 2025Using Text-Based Life Trajectories from Swedish Register Data to Predict Residential Mobility with Pretrained Transformers26 Nov 2025
- 2025Addressing Longstanding Challenges in Cognitive Science with Language Models31 Oct 2025
- 2025The Benchmarking Epistemology: Validity Theory for Evaluating Machine Learning Models27 Oct 2025
- 2025Deep Generative Model for Human Mobility Behavior7 Oct 2025
- 2025The Intercepted Self: How Generative AI Challenges the Dynamics of the Relational Self16 Sep 2025
- 2025ASCENDgpt: A Phenotype-Aware Transformer Model for Cardiovascular Risk Prediction from Electronic Health Records31 Aug 2025
- 2025Population-Scale Network Embeddings Expose Educational Divides in Network Structure Related to Right-Wing Populist Voting28 Aug 2025
12 newest of 27 All 27 (JSON)