# Assessing Social Alignment: Do Personality-Prompted Large Language Models Behave Like Humans?

arXiv: https://arxiv.org/abs/2412.16772 · HTML: https://mlsanity.com/paper/2412.16772/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 23 references that are in the map.
- First posted on arXiv 21 Dec 2024.

## Builds on (most cited first, 6 of 6)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies](https://mlsanity.com/paper/2208.10264/) (2022)
- [Evaluating and Inducing Personality in Pre-trained Language Models](https://mlsanity.com/paper/2206.07550/) (2022)
- [How Well Can LLMs Negotiate? NegotiationArena Platform and Analysis](https://mlsanity.com/paper/2402.05863/) (2024)
- [LLMs with Personalities in Multi-issue Negotiation Games](https://mlsanity.com/paper/2405.05248/) (2024)
- [Is persona enough for personality? Using ChatGPT to reconstruct an agent's latent personality from simple descriptions](https://mlsanity.com/paper/2406.12216/) (2024)

## Built on by (most cited first, 8 of 8)

- [Unmasking Conversational Bias in AI Multiagent Systems](https://mlsanity.com/paper/2501.14844/) (2025)
- [The Effect of Belief Boxes and Open-mindedness on Persuasion](https://mlsanity.com/paper/2512.06573/) (2025)
- [Can LLMs Think Like Consumers? Benchmarking Crowd-Level Reaction Reconstruction with ConsumerSimBench](https://mlsanity.com/paper/2605.17079/) (2026)
- [Position: Behavioral Systems Require Behavioral Tests](https://mlsanity.com/paper/2608.18081/) (2026)
- [Mimetic Alignment with ASPECT: Evaluation of AI-inferred Personal Profiles](https://mlsanity.com/paper/2603.26922/) (2026)
- [Personality, Role, and Expressive Style in Large Language Models: An Interactionist Analysis](https://mlsanity.com/paper/2605.28037/) (2026)
- [AI Alignment through a Game-theoretic Lens: A Survey](https://mlsanity.com/paper/2608.27910/) (2026)
- [Polite but Misaligned: Evaluating LLM Politeness Judgments Against Human Pragmatic Norms](https://mlsanity.com/paper/2609.29001/) (2026)

## Abstract (arXiv)

The ongoing revolution in language modeling has led to various novel applications, some of which rely on the emerging social abilities of large language models (LLMs). Already, many turn to the new cyber friends for advice during the pivotal moments of their lives and trust them with the deepest secrets, implying that accurate shaping of the LLM's personality is paramount. To this end, state-of-the-art approaches exploit a vast variety of training data, and prompt the model to adopt a particular personality. We ask (i) if personality-prompted models behave (i.e., make decisions when presented with a social situation) in line with the ascribed personality (ii) if their behavior can be finely controlled. We use classic psychological experiments, the Milgram experiment and the Ultimatum Game, as social interaction testbeds and apply personality prompting to open- and closed-source LLMs from 4 different vendors. Our experiments reveal failure modes of the prompt-based modulation of the models' behavior that are shared across all models tested and persist under prompt perturbations. These findings challenge the optimistic sentiment toward personality prompting generally held in the community.