# The average-farmer illusion in language-model simulations of agricultural decisions

arXiv: https://arxiv.org/abs/2609.15038 · HTML: https://mlsanity.com/paper/2609.15038/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use.
- Builds on 26 of its 53 references that are in the map.
- First posted on arXiv 14 Sep 2026.

## Placed in

- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 26 of 26)

- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Large Language Models Can Be Easily Distracted by Irrelevant Context](https://mlsanity.com/paper/2302.00093/) (2023)
- [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022)
- [Kimi K2: Open Agentic Intelligence](https://mlsanity.com/paper/2507.20534/) (2025)
- [State of What Art? A Call for Multi-Prompt LLM Evaluation](https://mlsanity.com/paper/2401.00595/) (2023)
- [LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals](https://mlsanity.com/paper/2411.10109/) (2024)
- [AgentSociety: Large-Scale Simulation of LLM-Driven Generative Agents Advances Understanding of Human Behaviors and Society](https://mlsanity.com/paper/2502.08691/) (2025)
- [Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs](https://mlsanity.com/paper/2502.17424/) (2025)
- [What Large Language Models Know and What People Think They Know](https://mlsanity.com/paper/2401.13835/) (2024)
- [Sensitivity, Performance, Robustness: Deconstructing the Effect of Sociodemographic Prompting](https://mlsanity.com/paper/2309.07034/) (2023)
- [Benchmarking large language models for biomedical natural language processing applications and recommendations](https://mlsanity.com/paper/2305.16326/) (2023)
- [Large Language Models for Disease Diagnosis: A Scoping Review](https://mlsanity.com/paper/2409.00097/) (2024)
- [Principled Personas: Defining and Measuring the Intended Effects of Persona Prompting on Task Performance](https://mlsanity.com/paper/2508.19764/) (2025)
- [PMark: Towards Robust and Distortion-free Semantic-level Watermarking with Channel Constraints](https://mlsanity.com/paper/2509.21057/) (2025)
- [Large language models as uncertainty-calibrated optimizers for experimental discovery](https://mlsanity.com/paper/2504.06265/) (2025)
- [Data Generation Using Large Language Models for Text Classification: An Empirical Case Study](https://mlsanity.com/paper/2407.12813/) (2024)
- [Variance reduction in output from generative AI](https://mlsanity.com/paper/2503.01033/) (2025)
- [AI Agent Behavioral Science](https://mlsanity.com/paper/2506.06366/) (2025)
- [Leveraging LLM-based agents for social science research: insights from citation network simulations](https://mlsanity.com/paper/2511.03758/) (2025)
- [Is More Context Always Better? Examining LLM Reasoning Capability for Time Interval Prediction](https://mlsanity.com/paper/2601.10132/) (2026)
- [Large Language Models Generate Harmful Responses Using a Distinct Mechanism, Shared Across Harm Types](https://mlsanity.com/paper/2604.09544/) (2026)
- [From Text to Insight: Leveraging Large Language Models for Performance Evaluation in Management](https://mlsanity.com/paper/2408.05328/) (2024)
- [Context-Value-Action Architecture for Value-Driven Large Language Model Agents](https://mlsanity.com/paper/2604.05939/) (2026)
- [KISS - Knowledge Infrastructure for Scientific Simulation: A Scaffolding for Agentic Earth Science](https://mlsanity.com/paper/2605.17856/) (2026)

## Abstract (arXiv)

Language-model agents are increasingly used as synthetic people in surveys and social simulations, yet their apparent realism is often judged from population averages or distributional similarity. We tested what such evidence actually establishes by comparing Claude, Codex and Kimi under four prespecified prompt designs with matched farmer decisions from China and four African countries. Some configurations reproduced observed means and adoption rates. However, their person-level predictions were weak; their decisions clustered around typical values and policy-relevant extremes were largely missing. Most strikingly, a simple generator fitted only to the observed marginal dis- tribution, and given no information about any farmer, achieved greater distributional similarity than every language-model configuration. Prompt additions produced conditional gains rather than uni- versal improvement: results varied with model, outcome, population and validation target. We call this the average-farmer illusion: a synthetic population can look realistic while failing to repro- duce who does what or how behaviour varies. We provide a claim-matched validation framework and reusable modular prompts that turn prompt construction into an auditable experimental process. Population-level resemblance should therefore be treated as the start of validation, not as evidence of individual simulation.