# Cognitive Models as Simulators: The Case of Moral Decision-Making

arXiv: https://arxiv.org/abs/2210.04121 · HTML: https://mlsanity.com/paper/2210.04121/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 60 references that are in the map.
- First posted on arXiv 8 Oct 2022.

## Builds on (most cited first, 4 of 4)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Self-supervised Pretraining of Visual Features in the Wild](https://mlsanity.com/paper/2103.01988/) (2021)
- [Over-representation of Extreme Events in Decision-Making: A Rational Metacognitive Account](https://mlsanity.com/paper/1801.09848/) (2018)

## Built on by (most cited first, 2 of 2)

- [Towards the Scalable Evaluation of Cooperativeness in Language Models](https://mlsanity.com/paper/2303.13360/) (2023)
- [AgentForge: A Flexible Low-Code Platform for Reinforcement Learning Agent Design](https://mlsanity.com/paper/2410.19528/) (2024)

## Abstract (arXiv)

To achieve desirable performance, current AI systems often require huge amounts of training data. This is especially problematic in domains where collecting data is both expensive and time-consuming, e.g., where AI systems require having numerous interactions with humans, collecting feedback from them. In this work, we substantiate the idea of textitcognitive models as simulators, which is to have AI systems interact with, and collect feedback from, cognitive models instead of humans, thereby making their training process both less costly and faster. Here, we leverage this idea in the context of moral decision-making, by having reinforcement learning (RL) agents learn about fairness through interacting with a cognitive model of the Ultimatum Game (UG), a canonical task in behavioral and brain sciences for studying fairness. Interestingly, these RL agents learn to rationally adapt their behavior depending on the emotional state of their simulated UG responder. Our work suggests that using cognitive models as simulators of humans is an effective approach for training AI systems, presenting an important way for computational cognitive science to make contributions to AI.