# Econometrics with Pre-Trained Embeddings for Unstructured Data

arXiv: https://arxiv.org/abs/2607.17378 · HTML: https://mlsanity.com/paper/2607.17378/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 74 references that are in the map.
- First posted on arXiv 19 Jul 2026.

## Builds on (most cited first, 15 of 15)

- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution](https://mlsanity.com/paper/2202.10054/) (2022)
- [DoubleMLDeep: Estimation of Causal Effects with Multimodal Data](https://mlsanity.com/paper/2402.01785/) (2024)
- [Inference for Regression with Variables Generated by AI or Machine Learning](https://mlsanity.com/paper/2402.15585/) (2024)
- [Causal Inference on Outcomes Learned from Text](https://mlsanity.com/paper/2503.00725/) (2025)
- [A Unifying Framework for Robust and Efficient Inference with Unstructured Data](https://mlsanity.com/paper/2505.00282/) (2025)
- [Adjustment for Confounding using Pre-Trained Representations](https://mlsanity.com/paper/2506.14329/) (2025)
- [The Fragility of Sparsity](https://mlsanity.com/paper/2311.02299/) (2023)
- [Program Evaluation with Remotely Sensed Outcomes](https://mlsanity.com/paper/2411.10959/) (2024)
- [Adventures in Demand Analysis Using AI](https://mlsanity.com/paper/2501.00382/) (2024)
- [Interpretable Discovery from Unstructured Data: A High-Dimensional Approach](https://mlsanity.com/paper/2511.01680/) (2025)
- [Copyright and Competition: Estimating Supply and Demand with Unstructured Data](https://mlsanity.com/paper/2501.16120/) (2025)
- [Demand Estimation with Text and Image Data](https://mlsanity.com/paper/2503.20711/) (2025)
- [Training and Testing with Multiple Splits: A Central Limit Theorem for Split-Sample Estimators](https://mlsanity.com/paper/2511.04957/) (2025)
- [From Unstructured Data to Demand Counterfactuals: Theory and Practice](https://mlsanity.com/paper/2601.05374/) (2026)

## Abstract (arXiv)

Unstructured data, such as images and text, are increasingly used in empirical economics. Since training machine-learning models on unstructured data is costly, economists often use off-the-shelf pre-trained deep learning models developed by computer scientists to extract embeddings, which are then used as covariates in target economic analyses. Despite the popularity of this practice, its theoretical foundations remain limited. There are two main difficulties. First, pre-trained models are typically trained on different datasets and for different tasks, making it unclear when they can be used reliably for the target task. Second, the embedding function is subject to an identification problem, complicating the analysis of its estimation error and the effect of that error on the target task. We provide sufficient conditions to overcome these difficulties. A key condition, which we call transferability, governs the convergence rate we derive. To assess transferability, we develop a computationally feasible bootstrap test that does not require re-estimating the embeddings and nuisance functions. Our theory applies to a wide range of double machine learning applications, including partially linear regression with unstructured controls, price elasticity estimation in demand models accounting for product quality measured by images and text, missing-data imputation using unstructured data, and average treatment effect estimation with unstructured confounders. As an empirical application, we estimate the labor supply elasticity on Amazon Mechanical Turk, an online labor market platform, using job-description embeddings as controls.