# auto-psych: Automating the science of mind using agent-driven theory discovery and experimentation

arXiv: https://arxiv.org/abs/2606.26460 · HTML: https://mlsanity.com/paper/2606.26460/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 66 references that are in the map.
- First posted on arXiv 24 Jun 2026.

## Builds on (most cited first, 25 of 25)

- [AlphaEvolve: A coding agent for scientific and algorithmic discovery](https://mlsanity.com/paper/2506.13131/) (2025)
- [ChemCrow: Augmenting large-language models with chemistry tools](https://mlsanity.com/paper/2304.05376/) (2023)
- [Agent Laboratory: Using LLM Agents as Research Assistants](https://mlsanity.com/paper/2501.04227/) (2025)
- [Accelerating scientific discovery with Co-Scientist](https://mlsanity.com/paper/2502.18864/) (2025)
- [Pyro: Deep Universal Probabilistic Programming](https://mlsanity.com/paper/1810.09538/) (2018)
- [AIDE: AI-Driven Exploration in the Space of Code](https://mlsanity.com/paper/2502.13138/) (2025)
- [Towards End-to-End Automation of AI Research](https://mlsanity.com/paper/2606.15497/) (2026)
- [Hypothesis Search: Inductive Reasoning with Language Models](https://mlsanity.com/paper/2309.05660/) (2023)
- [Empowering Biomedical Discovery with AI Agents](https://mlsanity.com/paper/2404.02831/) (2024)
- [Variational Bayesian Optimal Experimental Design](https://mlsanity.com/paper/1903.05480/) (2019)
- [Automated Statistical Model Discovery with Language Models](https://mlsanity.com/paper/2402.17879/) (2024)
- [EXP-Bench: Can AI Conduct AI Research Experiments?](https://mlsanity.com/paper/2505.24785/) (2025)
- [AutoDiscovery: Open-ended Scientific Discovery via Bayesian Surprise](https://mlsanity.com/paper/2507.00310/) (2025)
- [Evaluating Sakana's AI Scientist: Bold Claims, Mixed Results, and a Promising Future?](https://mlsanity.com/paper/2502.14297/) (2025)
- [Automating the Practice of Science -- Opportunities, Challenges, and Implications](https://mlsanity.com/paper/2409.05890/) (2024)
- [BoxingGym: Benchmarking Progress in Automated Experimental Design and Model Discovery](https://mlsanity.com/paper/2501.01540/) (2025)
- [Generating Computational Cognitive Models using Large Language Models](https://mlsanity.com/paper/2502.00879/) (2025)
- [Are Large Language Models Reliable AI Scientists? Assessing Reverse-Engineering of Black-Box Systems](https://mlsanity.com/paper/2505.17968/) (2025)
- [AutoScientists: Self-Organizing Agent Teams for Long-Running Scientific Experimentation](https://mlsanity.com/paper/2605.28655/) (2026)
- [Naturalistic Computational Cognitive Science: Towards generalizable models and theories that capture the full range of natural behavior](https://mlsanity.com/paper/2502.20349/) (2025)
- [AI-Assisted Discovery of Quantitative and Formal Models in Social Science](https://mlsanity.com/paper/2210.00563/) (2022)
- [Practical optimal experiment design with probabilistic programs](https://mlsanity.com/paper/1608.05046/) (2016)
- [CriticAL: Critic Automation with Language Models](https://mlsanity.com/paper/2411.06590/) (2024)
- [Toward an automated science of the mind](https://mlsanity.com/paper/2603.20988/) (2026)
- [ATLAS: Active Theory Learning for Automated Science](https://mlsanity.com/paper/2606.12386/) (2026)

## Built on by (most cited first, 5 of 5)

- [Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models](https://mlsanity.com/paper/2608.09696/) (2026)
- [BrainPilot: Automating Brain Discovery with Agentic Research](https://mlsanity.com/paper/2607.15079/) (2026)
- [Sparks of In Silico Cognitive Science: Theories from Simulated Data Can Generalize to Humans](https://mlsanity.com/paper/2609.08003/) (2026)
- [CogGym: Towards Large-Scale Comparative Evaluation of Human and Machine Cognition](https://mlsanity.com/paper/2609.21259/) (2026)
- [Hypothesis-guided discovery of cognitive algorithms via program refinement](https://mlsanity.com/paper/2610.02523/) (2026)

## Abstract (arXiv)

AI-based scientific automation is increasingly possible by using agents to generate hypotheses, design experiments, and analyze data. Data collection is a major bottleneck in this pipeline, however. Psychology, and computational cognitive science in particular, is well-positioned to benefit from AI experimentation because theories are often represented as code and crowdsourcing platforms enable programmatic human data collection at scale. Here, we apply automated discovery techniques to the project of generating theories in computational cognitive science, with an agent-based system collecting human data independently through crowdsourced survey experiments. As a testbed, we use a classic case study from cognitive psychology: judging which sequences of coin flips seem subjectively more random. Our system, auto-psych, uses nested agent-based discovery loops to generate explanatory theories of human behavior. The inner loop conjectures, fits, and critiques probabilistic cognitive models; the outer loop designs experiments to test these models, launches them online, and analyzes the data. This system can quickly and reliably recover ground-truth theories from synthetic data via systematic experimentation, but the nested structure is critical to model performance. Further, in three independent sequences of human experiments, the system finds theories that fit the data better than theories generated from the scientific literature. This work thus demonstrates the feasibility of automated data collection and theory discovery in computational cognitive science.