# Temperature and Persona Shape LLM Agent Consensus With Minimal Accuracy Gains in Qualitative Coding arXiv: https://arxiv.org/abs/2507.11198 ยท HTML: https://mlsanity.com/paper/2507.11198/ ## Key facts - Cited by 3 papers in the map (data through 14 Oct 2026). - Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. - Builds on 33 of its 86 references that are in the map. - First posted on arXiv 15 Jul 2025. ## Placed in - LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/) ## Builds on (most cited first, 30 of 33) - [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023) - [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023) - [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023) - [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://mlsanity.com/paper/2404.14219/) (2024) - [LLM Evaluators Recognize and Favor Their Own Generations](https://mlsanity.com/paper/2404.13076/) (2024) - [Multi-Agent Collaboration Mechanisms: A Survey of LLMs](https://mlsanity.com/paper/2501.06322/) (2025) - [LLMs Get Lost In Multi-Turn Conversation](https://mlsanity.com/paper/2505.06120/) (2025) - [A Survey of Hallucination in Large Foundation Models](https://mlsanity.com/paper/2309.05922/) (2023) - [ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs](https://mlsanity.com/paper/2309.13007/) (2023) - [PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traits](https://mlsanity.com/paper/2305.02547/) (2023) - [Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology View](https://mlsanity.com/paper/2310.02124/) (2023) - [Large Language Models as General Pattern Machines](https://mlsanity.com/paper/2307.04721/) (2023) - [Personality Traits in Large Language Models](https://mlsanity.com/paper/2307.00184/) (2023) - [Supporting Qualitative Analysis with Large Language Models: Combining Codebook with GPT-3 for Deductive Coding](https://mlsanity.com/paper/2304.10548/) (2023) - [LLM Generated Persona is a Promise with a Catch](https://mlsanity.com/paper/2503.16527/) (2025) - [LLM-Assisted Content Analysis: Using Large Language Models to Support Deductive Coding](https://mlsanity.com/paper/2306.14924/) (2023) - [Do LLMs Possess a Personality? Making the MBTI Test an Amazing Evaluation for Large Language Models](https://mlsanity.com/paper/2307.16180/) (2023) - [Multi-Agent Consensus Seeking via Large Language Models](https://mlsanity.com/paper/2310.20151/) (2023) - [Internal Consistency and Self-Feedback in Large Language Models: A Survey](https://mlsanity.com/paper/2407.14507/) (2024) - [Self-Assessment Tests are Unreliable Measures of LLM Personality](https://mlsanity.com/paper/2309.08163/) (2023) - [Position: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces!](https://mlsanity.com/paper/2504.09762/) (2025) - [Open Models, Closed Minds? On Agents Capabilities in Mimicking Human Personalities through Open Large Language Models](https://mlsanity.com/paper/2401.07115/) (2024) - [Can Large Language Models Serve as Data Analysts? A Multi-Agent Assisted Approach for Qualitative Data Analysis](https://mlsanity.com/paper/2402.01386/) (2024) - [Personality-Driven Decision-Making in LLM-Based Autonomous Agents](https://mlsanity.com/paper/2504.00727/) (2025) - [Spontaneous Emergence of Agent Individuality through Social Interactions in LLM-Based Communities](https://mlsanity.com/paper/2411.03252/) (2024) - [Automating Thematic Analysis: How LLMs Analyse Controversial Topics](https://mlsanity.com/paper/2405.06919/) (2024) - [TutorGym: A Testbed for Evaluating AI Agents as Tutors and Students](https://mlsanity.com/paper/2505.01563/) (2025) - [PromptHive: Bringing Subject Matter Experts Back to the Forefront with Collaborative Prompt Engineering for Educational Content Creation](https://mlsanity.com/paper/2410.16547/) (2024) - [Improving Student Learning with Hybrid Human-AI Tutoring: A Three-Study Quasi-Experimental Investigation](https://mlsanity.com/paper/2312.11274/) (2023) - [Single-Agent vs. Multi-Agent LLM Strategies for Automated Student Reflection Assessment](https://mlsanity.com/paper/2504.05716/) (2025) ## Built on by (most cited first, 3 of 3) - [Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not Ground Truth](https://mlsanity.com/paper/2607.28890/) (2026) - [Disagreement as Data: Reasoning Trace Analytics in Multi-Agent Systems](https://mlsanity.com/paper/2601.12618/) (2026) - [When Models Disagree: Rethinking LLM Evaluation for Public Comment Analysis](https://mlsanity.com/paper/2605.29025/) (2026) ## Abstract (arXiv) Large Language Models (LLMs) enable new possibilities for qualitative research at scale, including annotation and qualitative coding of educational data. While LLM-based multi-agent systems (MAS) can emulate human coding workflows, their benefits over single LLM agents for coding remain poorly understood. To that end, we conducted an experimental study of how persona and temperature of component agents of a MAS shapes consensus-building and coding accuracy for dialog segments. LLMs were prompted to code these segments deductively using a mature codebook with 8 codes and high inter-rater reliability derived from prior research. Our open-source MAS mirrors deductive human coding through structured agent discussion and consensus arbitration. Using six open-source LLMs (with 3 to 32 billion parameters) and 18 experimental configurations, we analyze over 77,000 coding decisions against a gold-standard dataset of human-annotated transcripts from online math tutoring sessions facilitated by educational software. Temperature significantly impacted whether and when consensus was reached across all six LLMs. MAS with multiple personas (including neutral, assertive, or empathetic) significantly delayed consensus in four out of six LLMs compared to uniform personas. In three of those LLMs, higher temperatures significantly diminished the effects of multiple personas on consensus. However, neither temperature nor persona pairing led to robust improvements in coding accuracy. Single agents matched or outperformed MAS consensus in most conditions. Qualitative analysis of MAS collaboration and coding disagreement may, however, improve codebook design and human-AI coding.