# Small Foundation Models of Human Cognition and Behaviour

arXiv: https://arxiv.org/abs/2608.05224 · HTML: https://mlsanity.com/paper/2608.05224/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 104 references that are in the map.
- First posted on arXiv 5 Aug 2026.

## Builds on (most cited first, 19 of 19)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Mixtral of Experts](https://mlsanity.com/paper/2401.04088/) (2024)
- [SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training](https://mlsanity.com/paper/2501.17161/) (2025)
- [2 OLMo 2 Furious](https://mlsanity.com/paper/2501.00656/) (2024)
- [SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model](https://mlsanity.com/paper/2502.02737/) (2025)
- [Llama-Nemotron: Efficient Reasoning Models](https://mlsanity.com/paper/2505.00949/) (2025)
- [Olmo 3](https://mlsanity.com/paper/2512.13961/) (2025)
- [Nemotron-H: A Family of Accurate and Efficient Hybrid Mamba-Transformer Models](https://mlsanity.com/paper/2504.03624/) (2025)
- [Hermes 3 Technical Report](https://mlsanity.com/paper/2408.11857/) (2024)
- [Generating Computational Cognitive Models using Large Language Models](https://mlsanity.com/paper/2502.00879/) (2025)
- [Using Reinforcement Learning to Train Large Language Models to Explain Human Decisions](https://mlsanity.com/paper/2505.11614/) (2025)
- [Be.FM: Open Foundation Models for Human Behavior](https://mlsanity.com/paper/2505.23058/) (2025)
- [Hermes 4 Technical Report](https://mlsanity.com/paper/2508.18255/) (2025)
- [HumanLLM: Towards Personalized Understanding and Simulation of Human Nature](https://mlsanity.com/paper/2601.15793/) (2026)
- [Bayesian Statistical Modeling with Predictors from LLMs](https://mlsanity.com/paper/2406.09012/) (2024)
- [Not Yet AlphaFold for the Mind: Evaluating Centaur as a Synthetic Participant](https://mlsanity.com/paper/2508.07887/) (2025)
- [Post-training makes large language models less human-like](https://mlsanity.com/paper/2605.07632/) (2026)
- [Not Even Wrong: On the Limits of Prediction as Explanation in Cognitive Science](https://mlsanity.com/paper/2510.03311/) (2025)

## Built on by (most cited first, 1 of 1)

- [Better Behavioral Prediction, More Faithful Model Ablations? Evidence from Sequential Choice](https://mlsanity.com/paper/2609.36097/) (2026)

## Abstract (arXiv)

Large language models fine-tuned on human behavioural data have emerged as general-purpose cognitive proxies, but the scale this requires, and whether these models process task structure or exploit statistical shortcuts, remain open questions. We train fourteen models from 135M to 14B parameters across four architecture families on Psych-101, a dataset of 10.7 million trial-level choices from 160 experiments. For in-distribution simulations, scale barely matters. The models fall within a narrow band, as though against a ceiling, and 0.6B to 1B parameters suffice to match a 70B baseline on held-out participants. Out-of-distribution, that band opens into a markedly steeper scaling gradient, with larger models clearly advantaged in generalisation to novel task structure. To determine what information these models use, we run two diagnostics. We progressively strip four prompt channels -- task instructions, experimental stimuli, outcome feedback, and choice history -- across 27 experiments, and permute trial order. Masking the content of stimuli and feedback destroys 75.7% of learned information and pushes models below chance, demonstrating that choice history alone does not account for performance. Permutation reveals invariance on tasks with independent trials but sensitivity where trial order is determined by prior responses. Small cognitively fine-tuned models therefore show promise as noise ceiling estimators for psychological experiments, though their scope remains bounded by the paradigms seen in training.