# PICon: A Multi-Turn Interrogation Framework for Evaluating Persona Agent Consistency

arXiv: https://arxiv.org/abs/2603.25620 · HTML: https://mlsanity.com/paper/2603.25620/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use.
- Builds on 18 of its 30 references that are in the map.
- First posted on arXiv 26 Mar 2026.

## Placed in

- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 18 of 18)

- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [Personalizing Dialogue Agents: I have a dog, do you have pets too?](https://mlsanity.com/paper/1801.07243/) (2018)
- [Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies](https://mlsanity.com/paper/2208.10264/) (2022)
- [LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals](https://mlsanity.com/paper/2411.10109/) (2024)
- [InCharacter: Evaluating Personality Fidelity in Role-Playing Agents through Psychological Interviews](https://mlsanity.com/paper/2310.17976/) (2023)
- [Dialogue Natural Language Inference](https://mlsanity.com/paper/1811.00671/) (2018)
- [PersonaGym: Evaluating Persona Agents and LLMs](https://mlsanity.com/paper/2407.18416/) (2024)
- [LLM Generated Persona is a Promise with a Catch](https://mlsanity.com/paper/2503.16527/) (2025)
- [I like fish, especially dolphins: Addressing Contradictions in Dialogue Modeling](https://mlsanity.com/paper/2012.13391/) (2020)
- [Consistently Simulating Human Personas with Multi-Turn Reinforcement Learning](https://mlsanity.com/paper/2511.00222/) (2025)
- [Evaluating Character Understanding of Large Language Models via Character Profiling from Fictional Works](https://mlsanity.com/paper/2404.12726/) (2024)
- [PatientSim: A Persona-Driven Simulator for Realistic Doctor-Patient Interactions](https://mlsanity.com/paper/2505.17818/) (2025)
- [OpenCharacter: Training Customizable Role-Playing LLMs with Large-Scale Synthetic Personas](https://mlsanity.com/paper/2501.15427/) (2025)
- [Profile Consistency Identification for Open-domain Dialogue Agents](https://mlsanity.com/paper/2009.09680/) (2020)
- [Twin-2K-500: A dataset for building digital twins of over 2,000 people based on their answers to over 500 questions](https://mlsanity.com/paper/2505.17479/) (2025)
- [Human Simulacra: Benchmarking the Personification of Large Language Models](https://mlsanity.com/paper/2402.18180/) (2024)
- [DeepPersona: A Generative Engine for Scaling Deep Synthetic Personas](https://mlsanity.com/paper/2511.07338/) (2025)
- [Spotting Out-of-Character Behavior: Atomic-Level Evaluation of Persona Fidelity in Open-Ended Generation](https://mlsanity.com/paper/2506.19352/) (2025)

## Built on by (most cited first, 3 of 3)

- [RealUserSim: Bridging the Reality Gap in Agent Benchmarking via Grounded User Simulation](https://mlsanity.com/paper/2605.20204/) (2026)
- [NeMo Data Designer: An Extensible Framework for Multimodal Synthetic Data Generation](https://mlsanity.com/paper/2609.17699/) (2026)
- [Measuring and Improving Behavioral Consistency in Large Language Models through Fact-Heuristic-Emotion State Enforcement](https://mlsanity.com/paper/2607.24765/) (2026)

## Abstract (arXiv)

Large language model (LLM)-based persona agents are rapidly being adopted as scalable proxies for human participants across diverse domains. Yet there is no systematic method for verifying whether a persona agent's responses remain free of contradictions and factual inaccuracies throughout an interaction. A principle from interrogation methodology offers a lens: no matter how elaborate a fabricated identity, systematic interrogation will expose its contradictions. We apply this principle to propose PICon, an evaluation framework that probes persona agents through logically chained multi-turn questioning. PICon evaluates consistency along three core dimensions: internal consistency (freedom from self-contradiction), external consistency (alignment with real-world facts), and retest consistency (stability under repetition). Evaluating seven groups of persona agents alongside 63 real human participants, we find that even systems previously reported as highly consistent fail to meet the human baseline across all three dimensions, revealing contradictions and evasive responses under chained questioning. This work provides both a conceptual foundation and a practical methodology for evaluating persona agents before trusting them as substitutes for human participants. We provide the source code and an interactive demo at: https://kaist-edlab.github.io/picon/