# Large Language Models: An Applied Econometric Framework

arXiv: https://arxiv.org/abs/2412.07031 · HTML: https://mlsanity.com/paper/2412.07031/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Dec 2024.

## Built on by (most cited first, 30 of 32)

- [The Alternative Annotator Test for LLM-as-a-Judge: How to Statistically Justify Replacing Human Annotators with LLMs](https://mlsanity.com/paper/2501.10970/) (2025)
- [This human study did not involve human subjects: Validating LLM simulations as behavioral evidence](https://mlsanity.com/paper/2602.15785/) (2026)
- [Large Language Models for Market Research: A Data-augmentation Approach](https://mlsanity.com/paper/2412.19363/) (2024)
- [Causal Inference on Outcomes Learned from Text](https://mlsanity.com/paper/2503.00725/) (2025)
- [Replayable Financial Agents: A Determinism-Faithfulness Assurance Harness for Tool-Using LLM Agents](https://mlsanity.com/paper/2601.15322/) (2026)
- [Prompting for Policy: Forecasting Macroeconomic Scenarios with Synthetic LLM Personas](https://mlsanity.com/paper/2511.02458/) (2025)
- [Scaling Point-in-Time Language Models: Enabling Economic Evaluation of Embeddings](https://mlsanity.com/paper/2607.11889/) (2026)
- [The Measurement Revolution? Credible Measurement and Inference in the Age of AI](https://mlsanity.com/paper/2608.23524/) (2026)
- [Behavioral Generative Agents for Energy Operations](https://mlsanity.com/paper/2506.12664/) (2025)
- [Inflation Attitudes of Large Language Models](https://mlsanity.com/paper/2512.14306/) (2025)
- [Statistical realism is not evidence that LLMs can estimate treatment effects in social science experiments](https://mlsanity.com/paper/2604.02458/) (2026)
- [LLM Predictive Scoring and Validation: Inferring Experience Ratings from Unstructured Text](https://mlsanity.com/paper/2604.14321/) (2026)
- [The signal is the ceiling: Measurement limits of LLM-predicted experience ratings from open-ended survey text](https://mlsanity.com/paper/2604.19645/) (2026)
- [Best-Arm Identification with Generative Proxy](https://mlsanity.com/paper/2607.06879/) (2026)
- [Debiased Inference for AI-Generated Data without Gold-Standard Labels: Identification via Multiple Imperfect Measurements](https://mlsanity.com/paper/2608.18294/) (2026)
- [When Can LLM Digital Twins Reduce Human Measurement? From Behavioral Fidelity to Statistical Substitutability](https://mlsanity.com/paper/2609.07987/) (2026)
- [Ordinary Least Squares as an Attention Mechanism](https://mlsanity.com/paper/2504.09663/) (2025)
- [Epitome: Pioneering an Experimental Platform for AI-Social Science Integration](https://mlsanity.com/paper/2507.01061/) (2025)
- [Left Leaning Models: How AI Evaluates Economic Policy?](https://mlsanity.com/paper/2507.15771/) (2025)
- [How AI Forecasts AI Jobs: Benchmarking LLM Predictions of Labor Market Changes](https://mlsanity.com/paper/2510.23358/) (2025)
- [Foundation Priors](https://mlsanity.com/paper/2512.01107/) (2025)
- [From Model Choice to Model Belief: Establishing a New Measure for LLM-Based Research](https://mlsanity.com/paper/2512.23184/) (2025)
- [MemGuard-Alpha: Limits of Membership Inference for Detecting and Filtering Memorization-Contaminated Signals in LLM-Based Financial Forecasting](https://mlsanity.com/paper/2603.26797/) (2026)
- [Bootstrapping with AI/ML-generated labels](https://mlsanity.com/paper/2604.23770/) (2026)
- [Validating LLMs in social science: Epistemic threats and emerging norms](https://mlsanity.com/paper/2607.07915/) (2026)
- [Tabular Foundation Models for Discrete Choice Estimation](https://mlsanity.com/paper/2607.13314/) (2026)
- [When Predictions Become Regressors: A Split-Sample Correction for Biases in Downstream Inference](https://mlsanity.com/paper/2608.02909/) (2026)
- [Modeling Human Behavior with Type Vectors Using AI](https://mlsanity.com/paper/2608.18265/) (2026)
- [AI Contextual Measurement for Recovering Individual and Group-Level Effects: Validation Against Survey Measures and an Occupational Application](https://mlsanity.com/paper/2609.02821/) (2026)
- [From Preference to Reciprocity: Decentralized Matching with Empirically Grounded LLM-agent Based Modeling](https://mlsanity.com/paper/2609.34679/) (2026)

## Abstract (arXiv)

Large language models (LLMs) enable researchers to analyze text at unprecedented scale and minimal cost. Researchers can now revisit old questions and tackle novel ones with rich data. We provide an econometric framework for realizing this potential in two empirical uses. For prediction problems -- forecasting outcomes from text -- valid conclusions require ``no training leakage'' between the LLM's training data and the researcher's sample, which can be enforced through careful model choice and research design. For estimation problems -- automating the measurement of economic concepts for downstream analysis -- valid downstream inference requires combining LLM outputs with a small validation sample to deliver consistent and precise estimates. Absent a validation sample, researchers cannot assess possible errors in LLM outputs, and consequently seemingly innocuous choices (which model, which prompt) can produce dramatically different parameter estimates. When used appropriately, LLMs are powerful tools that can expand the frontier of empirical economics.