# Toward a principled Bayesian workflow in cognitive science

arXiv: https://arxiv.org/abs/1904.12765 · HTML: https://mlsanity.com/paper/1904.12765/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Apr 2019.

## Built on by (most cited first, 17 of 17)

- [BayesFlow: Amortized Bayesian Workflows With Neural Networks](https://mlsanity.com/paper/2306.16015/) (2023)
- [A Deep Learning Method for Comparing Bayesian Hierarchical Models](https://mlsanity.com/paper/2301.11873/) (2023)
- ["AI enhances our performance, I have no doubt this one will do the same": The Placebo effect is robust to negative descriptions of AI](https://mlsanity.com/paper/2309.16606/) (2023)
- [Simulation-based Bayesian inference under model misspecification](https://mlsanity.com/paper/2503.12315/) (2025)
- [BayesFlow 2: Multi-Backend Amortized Bayesian Inference in Python](https://mlsanity.com/paper/2602.07098/) (2026)
- [Neural Superstatistics for Bayesian Estimation of Dynamic Cognitive Models](https://mlsanity.com/paper/2211.13165/) (2022)
- [Pose Prior Learner: Unsupervised Categorical Prior Learning for Pose Estimation](https://mlsanity.com/paper/2410.03858/) (2024)
- [SEAM: An Integrated Activation-Coupled Model of Sentence Processing and Eye Movements in Reading](https://mlsanity.com/paper/2303.05221/) (2023)
- [Designing Optimal Behavioral Experiments Using Machine Learning](https://mlsanity.com/paper/2305.07721/) (2023)
- [Amortized Bayesian Mixture Models](https://mlsanity.com/paper/2501.10229/) (2025)
- [Posterior SBC: Simulation-Based Calibration Checking Conditional on Data](https://mlsanity.com/paper/2502.03279/) (2025)
- [From Mice to Trains: Amortized Bayesian Inference on Graph Data](https://mlsanity.com/paper/2601.02241/) (2026)
- [Exploring the anatomy of articulation rate in spontaneous English speech: relationships between utterance length effects and social factors](https://mlsanity.com/paper/2408.06732/) (2024)
- [Profit over Proxies: A Scalable Bayesian Decision Framework for Optimizing Multi-Variant Online Experiments](https://mlsanity.com/paper/2509.22677/) (2025)
- [Recovery-Directed Symbolic Distillation of Neural Likelihoods](https://mlsanity.com/paper/2609.32409/) (2026)
- [Mitigating Representation Gaps in Amortized Bayesian Inference with Auxiliary Supervision](https://mlsanity.com/paper/2609.39525/) (2026)
- [When the AI Leaves the Tailorshop: Measuring What an LLM Advisor Leaves Behind in Complex Problem Solving](https://mlsanity.com/paper/2610.00163/) (2026)

## Abstract (arXiv)

Experiments in research on memory, language, and in other areas of cognitive science are increasingly being analyzed using Bayesian methods. This has been facilitated by the development of probabilistic programming languages such as Stan, and easily accessible front-end packages such as brms. The utility of Bayesian methods, however, ultimately depends on the relevance of the Bayesian model, in particular whether or not it accurately captures the structure of the data and the data analyst's domain expertise. Even with powerful software, the analyst is responsible for verifying the utility of their model. To demonstrate this point, we introduce a principled Bayesian workflow (Betancourt, 2018) to cognitive science. Using a concrete working example, we describe basic questions one should ask about the model: prior predictive checks, computational faithfulness, model sensitivity, and posterior predictive checks. The running example for demonstrating the workflow is data on reading times with a linguistic manipulation of object versus subject relative clause sentences. This principled Bayesian workflow also demonstrates how to use domain knowledge to inform prior distributions. It provides guidelines and checks for valid data analysis, avoiding overfitting complex models to noise, and capturing relevant data structure in a probabilistic model. Given the increasing use of Bayesian methods, we aim to discuss how these methods can be properly employed to obtain robust answers to scientific questions. All data and code accompanying this paper are available from https://osf.io/b2vx9/.