Can Multi-Agent LLMs Identify Their Peers? Stylometric Fingerprinting in Role-Constrained Political Analysis
On the map: LLM agents and tool use → Social simulation and multi-agent debate
Abstract (arXiv)
Multi-agent large language model (LLM) pipelines for political statement analysis are vulnerable to peer-preservation bias: models tend to protect peer models from deactivation and show identity-dependent scoring distortions. Prompt-level anonymization was proposed as a mitigation, but prior work simultaneously documented that stylometric fingerprints survive anonymization in role-constrained outputs - raising the question of whether this mitigation is sufficient. This paper provides the first systematic investigation of whether LLMs can identify the model family behind political analysis texts under anonymization conditions. We evaluate three classifier approaches - LLM zero-shot and few-shot (Claude Sonnet 4.6 and Llama-3.3-70B) and a fine-tuned T5-base model - on a five-class attribution task covering four commercial LLM families and an open-world 'unknown' class. We introduce a statement-disjoint cross-validation protocol (SD-CV; defined in Section 3.5) that guarantees no content overlap between training and validation data, and contrast it with a run-disjoint baseline (RD-CV). T5 achieves Macro F1 = 0.991 (+-0.008) under SD-CV and F1 = 0.978 on 24 completely held-out statements - robust despite a 2.1x increase in train-test content distance versus RD-CV (0.767 vs. 0.366, p<0.001), demonstrating genuine stylometric generalization. A fractional SD-CV analysis identifies a performance knee at 40% of training data (~440 texts). Our findings confirm that prompt-level anonymization alone cannot neutralize model identity signals, with direct implications for EU AI Act compliance (Articles 13, 14, 26) and for computer system validation (CSV) in quality-critical multi-agent deployments.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 7 of its 78 references that are in the map. First posted on arXiv 20 May 2026.
Builds on
7 of 78 references.
Closest ancestors · 3
Same direction, newest first
- 2026When Roles Fail: Epistemic Constraints on Advocate Role Fidelity in LLM-Based Political Statement Analysis29 Apr
- 2026From Safety Risk to Design Principle: Peer-Preservation in Multi-Agent LLM Systems and Its Implications for Orchestrated Democratic Discourse Analysis9 Apr
- 2025When Identity Skews Debate: Anonymization for Bias-Reduced Multi-Agent Reasoning8 Oct
Other references · 4
- 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
- 2024Do LLMs write like humans? Variation in grammatical and rhetorical stylescited by 62
- 2026Peer-Preservation in Frontier Modelscited by 9
- 2025The Hidden DNA of LLM-Generated JavaScript: Structural Patterns Enable High-Accuracy Authorship Attributioncited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Peer Identity Bias in Multi-Agent LLM Evaluation: An Empirical Study Using the TRUST Democratic Discourse Analysis Pipeline
- 2026LLMs Can Infer Political Alignment from Online Conversations
- 2026How Far Will They Go? Red-Teaming Online Influence with Large Language Models
- 2024GermanPartiesQA: Benchmarking Commercial Large Language Models and AI Companions for Political Alignment and Sycophancy
- 2025Agent-Enhanced Large Language Models for Researching Political Institutions
- 2026Political Alignment in Large Language Models: A Multidimensional Audit of Psychometric Identity and Behavioral Bias
- 2026Agentic Framework for Political Biography Extraction
- 2026Large Language Models and Augmented Democracy