# Do LLMs Understand Personality? Rethinking Persona Fidelity Evaluation through Structured Behavioral Inference

arXiv: https://arxiv.org/abs/2608.26674 · HTML: https://mlsanity.com/paper/2608.26674/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 8 of its 62 references that are in the map.
- First posted on arXiv 27 Aug 2026.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 8 of 8)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models](https://mlsanity.com/paper/2512.02556/) (2025)
- [CharacterChat: Learning towards Conversational AI with Personalized Social Support](https://mlsanity.com/paper/2308.10278/) (2023)
- [ChatHaruhi: Reviving Anime Character in Reality via Large Language Model](https://mlsanity.com/paper/2308.09597/) (2023)
- [Large Language Model Psychometrics: A Systematic Review of Evaluation, Validation, and Enhancement](https://mlsanity.com/paper/2505.08245/) (2025)
- [OpenCharacter: Training Customizable Role-Playing LLMs with Large-Scale Synthetic Personas](https://mlsanity.com/paper/2501.15427/) (2025)
- [Rethinking Role-Playing Evaluation: Anonymous Benchmarking and a Systematic Study of Personality Effects](https://mlsanity.com/paper/2603.03915/) (2026)
- [PsyCLIENT: Client Simulation via Conversational Trajectory Modeling for Trainee Practice and Model Evaluation in Mental Health Counseling](https://mlsanity.com/paper/2601.07312/) (2026)

## Built on by (most cited first, 1 of 1)

- [ProMediConv: Benchmarking Proactive Conversational Agents in Legal Dispute Mediation](https://mlsanity.com/paper/2609.11101/) (2026)

## Abstract (arXiv)

As large language models are increasingly deployed to simulate diverse human characters, ensuring persona fidelity, defined as the extent to which an agent's behavior consistently reflects the psychological and stylistic characteristics of a target persona, has become a critical requirement. However, existing evaluation paradigms primarily rely on either holistic LLM-based judges, which are prone to "holistic appraisal hallucination'', or static psychometric inventories, which fail to capture the context-dependent fidelity required in dynamic dialogue. To address these limitations, we propose PRISM (Persona Reasoning with Inverse SFL-based Modeling), a psycholinguistically grounded framework that reformulates persona fidelity evaluation as a structured inverse inference task. Inspired by Systemic Functional Linguistics (SFL), PRISM decomposes persona fidelity into three functional dimensions: Task Framing, Interpersonal Stance, and Linguistic Style. It estimates dimension-specific evidence over a persona-conditioned label space and aggregates these signals into an interpretable and auditable evaluation process. Experiments show that PRISM yields more accurate and stable judgements than traditional holistic judging, providing a more reliable framework for persona fidelity evaluation.