mlsanityData to 14 Oct 2026

Persona Alchemy: Designing, Evaluating, and Implementing Psychologically-Grounded LLM Agents for Diverse Stakeholder Representation

arXiv 2505.18351 ↗First posted 23 May 2025cs.MA cs.CY cs.DBcited by 2

On the map: LLM agents and tool use → Social simulation and multi-agent debate

Abstract (arXiv)

Despite advances in designing personas for Large Language Models (LLM), challenges remain in aligning them with human cognitive processes and representing diverse stakeholder perspectives. We introduce a Social Cognitive Theory (SCT) agent design framework for designing, evaluating, and implementing psychologically grounded LLMs with consistent behavior. Our framework operationalizes SCT through four personal factors (cognitive, motivational, biological, and affective) for designing, six quantifiable constructs for evaluating, and a graph database-backed architecture for implementing stakeholder personas. Experiments tested agents' responses to contradicting information of varying reliability. In the highly polarized renewable energy transition discourse, we design five diverse agents with distinct ideologies, roles, and stakes to examine stakeholder representation. The evaluation of these agents in contradictory scenarios occurs through comprehensive processes that implement the SCT. Results show consistent response patterns (R2 range: 0.58-0.61) and systematic temporal development of SCT construct effects. Principal component analysis identifies two dimensions explaining 73% of variance, validating the theoretical structure. Our framework offers improved explainability and reproducibility compared to black-box approaches. This work contributes to ongoing efforts to improve diverse stakeholder representation while maintaining psychological consistency in LLM personas.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 18 of its 58 references that are in the map. First posted on arXiv 23 May 2025.

Builds on

18 of 58 references.

References · 18

  1. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,395
  2. 2023Whose Opinions Do Language Models Reflect?cited by 773
  3. 2023Zephyr: Direct Distillation of LM Alignmentcited by 506
  4. 2024Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalizationcited by 208
  5. 2023Personality Traits in Large Language Modelscited by 169
  6. 2024Quantifying the Persona Effect in LLM Simulationscited by 169
  7. 2022SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualizationcited by 168
  8. 2024Large language models that replace human participants can harmfully misportray and flatten identity groupscited by 144
  9. 2024Evaluating Large Language Model Biases in Persona-Steered Generationcited by 78
  10. 2024The illusion of artificial inclusioncited by 34
  11. 2024Modeling Human Subjectivity in LLMs Using Explicit and Implicit Human Factors in Personascited by 23
  12. 2023The Machine Psychology of Cooperation: Can GPT models operationalise prompts for altruism, cooperation, competitiveness and selfishness in economic games?cited by 22
6 more
  1. 2024CloChat: Understanding How People Customize, Interact, and Experience Personas in Large Language Modelscited by 19
  2. 2024Eliciting Personality Traits in Large Language Modelscited by 12
  3. 2024"In Dialogues We Learn": Towards Personalized Dialogue Without Pre-defined Profiles through In-Dialogue Learningcited by 10
  4. 2024Stark: Social Long-Term Multi-Modal Conversation with Persona Commonsense Knowledgecited by 8
  5. 2024MORPHEUS: Modeling Role from Personalized Dialogue History by Exploring and Utilizing Latent Spacecited by 5
  6. 2024Personality Differences Drive Conversational Dynamics: A High-Dimensional NLP Approachcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2023Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology View
  2. 2025The Social Laboratory: A Psychometric Framework for Multi-Agent LLM Evaluation
  3. 2026LLM-enabled Social Agents
  4. 2026From Survey Personas to LLM Agents: A Generative Agent-based Simulation of Mobility Policy Preference Dynamics
  5. 2025Applying Psychometrics to Large Language Model Simulated Populations: Recreating the HEXACO Personality Inventory Experiment with Generative Agents
  6. 2026Agent Sociodrama: Social-Dynamics-Guided Persona Recasting for LLM Agent Teams
  7. 2025From the Fluency Fallacy to the Micro-to-Macro Validity Gap: Opportunities and Pitfalls of LLMs in Social Simulation
  8. 2025Simulating Generative Social Agents via Theory-Informed Workflow Design