# CAREER: A Foundation Model for Labor Sequence Data

arXiv: https://arxiv.org/abs/2202.08370 · HTML: https://mlsanity.com/paper/2202.08370/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 66 references that are in the map.
- First posted on arXiv 16 Feb 2022.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection](https://mlsanity.com/paper/2004.07667/) (2020)
- [FinBERT: Financial Sentiment Analysis with Pre-trained Language Models](https://mlsanity.com/paper/1908.10063/) (2019)
- [Sequence Length is a Domain: Length-based Overfitting in Transformer Models](https://mlsanity.com/paper/2109.07276/) (2021)
- [Rationales for Sequential Predictions](https://mlsanity.com/paper/2109.06387/) (2021)
- [Job2Vec: Job Title Benchmarking with Collective Multi-View Representation Learning](https://mlsanity.com/paper/2009.07429/) (2020)
- [SHOPPER: A Probabilistic Model of Consumer Choice with Substitutes and Complements](https://mlsanity.com/paper/1711.03560/) (2017)
- [Counterfactual Inference for Consumer Choice Across Many Product Categories](https://mlsanity.com/paper/1906.02635/) (2019)

## Built on by (most cited first, 11 of 11)

- [Predictions as Surrogates: Revisiting Surrogate Outcomes in the Age of AI](https://mlsanity.com/paper/2501.09731/) (2025)
- [Causal Inference on Outcomes Learned from Text](https://mlsanity.com/paper/2503.00725/) (2025)
- [LABOR-LLM: Language-Based Occupational Representations with Large Language Models](https://mlsanity.com/paper/2406.17972/) (2024)
- [KARRIEREWEGE: A Large Scale Career Path Prediction Dataset](https://mlsanity.com/paper/2412.14612/) (2024)
- [Estimating Wage Disparities Using Foundation Models](https://mlsanity.com/paper/2409.09894/) (2024)
- [JobHop: A Large-Scale Dataset of Career Trajectories](https://mlsanity.com/paper/2505.07653/) (2025)
- [JobHop v2: A Large-Scale Career Trajectory Dataset from Unstructured Resumes](https://mlsanity.com/paper/2607.11715/) (2026)
- [STEP: Career-Path Recommendation via Temporal and Educational Trajectory Modeling](https://mlsanity.com/paper/2607.11722/) (2026)
- [The Book of Life approach: Enabling richness and scale for life course research](https://mlsanity.com/paper/2507.03027/) (2025)
- [LifeSentence: Language models can encode human life course trajectories from longitudinal panel data](https://mlsanity.com/paper/2606.11220/) (2026)
- [When Can We Work in Embedding Space? What Text Embeddings Preserve](https://mlsanity.com/paper/2608.31059/) (2026)

## Abstract (arXiv)

Labor economists regularly analyze employment data by fitting predictive models to small, carefully constructed longitudinal survey datasets. Although machine learning methods offer promise for such problems, these survey datasets are too small to take advantage of them. In recent years large datasets of online resumes have also become available, providing data about the career trajectories of millions of individuals. However, standard econometric models cannot take advantage of their scale or incorporate them into the analysis of survey data. To this end we develop CAREER, a foundation model for job sequences. CAREER is first fit to large, passively-collected resume data and then fine-tuned to smaller, better-curated datasets for economic inferences. We fit CAREER to a dataset of 24 million job sequences from resumes, and adjust it on small longitudinal survey datasets. We find that CAREER forms accurate predictions of job sequences, outperforming econometric baselines on three widely-used economics datasets. We further find that CAREER can be used to form good predictions of other downstream variables. For example, incorporating CAREER into a wage model provides better predictions than the econometric models currently in use.