mlsanityData to 7 Oct 2026

Pluralistic Behavior Suite: Stress-Testing Multi-Turn Adherence to Custom Behavioral Policies

arXiv 2511.05018 ↗First posted 7 Nov 2025cs.CL cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) are typically aligned to a universal set of safety and usage principles intended for broad public acceptability. Yet, real-world applications of LLMs often take place within organizational ecosystems shaped by distinctive corporate policies, regulatory requirements, use cases, brand guidelines, and ethical commitments. This reality highlights the need for rigorous and comprehensive evaluation of LLMs with pluralistic alignment goals, an alignment paradigm that emphasizes adaptability to diverse user values and needs. In this work, we present PLURALISTIC BEHAVIOR SUITE (PBSUITE), a dynamic evaluation suite designed to systematically assess LLMs' capacity to adhere to pluralistic alignment specifications in multi-turn, interactive conversations. PBSUITE consists of (1) a diverse dataset of 300 realistic LLM behavioral policies, grounded in 30 industries; and (2) a dynamic evaluation framework for stress-testing model compliance with custom behavioral specifications under adversarial conditions. Using PBSUITE, We find that leading open- and closed-source LLMs maintain robust adherence to behavioral policies in single-turn settings (less than 4% failure rates), but their compliance weakens substantially in multi-turn adversarial interactions (up to 84% failure rates). These findings highlight that existing model alignment and safety moderation methods fall short in coherently enforcing pluralistic behavioral policies in real-world LLM interactions. Our work contributes both the dataset and analytical framework to support future research toward robust and context-aware pluralistic alignment techniques.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 43 of its 58 references that are in the map. First posted on arXiv 7 Nov 2025.

Builds on

43 of 58 references.

References · 43

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  3. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  4. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  5. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  6. 2022BERTopic: Neural topic modeling with a class-based TF-IDF procedurecited by 779
  7. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
  8. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 533
  9. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 520
  10. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 478
  11. 2024WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMscited by 460
  12. 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 441
31 more
  1. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 364
  2. 2024Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Modelscited by 301
  3. 2016Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Modelscited by 299
  4. 2023ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMscited by 297
  5. 2024A Roadmap to Pluralistic Alignmentcited by 175
  6. 2024Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputscited by 148
  7. 2024LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks Yetcited by 145
  8. 2024The PRISM Alignment Dataset: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Modelscited by 136
  9. 2025Aegis2.0: A Diverse AI Safety Dataset and Risks Taxonomy for Alignment of LLM Guardrailscited by 122
  10. 2025X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agentscited by 112
  11. 2024Modular Pluralism: Pluralistic Alignment via Multi-LLM Collaborationcited by 111
  12. 2024Chain of Attack: a Semantic-Driven Contextual Multi-Turn attacker for LLMcited by 73
  13. 2024PERSONA: A Reproducible Testbed for Pluralistic Alignmentcited by 71
  14. 2024Speak Out of Turn: Safety Vulnerability of Large Language Models in Multi-turn Dialoguecited by 64
  15. 2025Reasoning-Augmented Conversation for Multi-Turn Jailbreak Attacks on Large Language Modelscited by 61
  16. 2025RealSafe-R1: Safety-Aligned DeepSeek-R1 without Compromising Reasoning Capabilitycited by 58
  17. 2024One fish, two fish, but not the whole sea: Alignment reduces language models' conceptual diversitycited by 57
  18. 2024Multi-Turn Context Jailbreak Attack on Large Language Models From First Principlescited by 54
  19. 2024PAL: Pluralistic Alignment Framework for Learning from Heterogeneous Preferencescited by 53
  20. 2025Foot-In-The-Door: A Multi-turn Jailbreak for LLMscited by 50
  21. 2022CodeAttack: Code-Based Adversarial Attacks for Pre-trained Programming Language Modelscited by 48
  22. 2025Modifying Large Language Model Post-Training for Diverse Creative Writingcited by 39
  23. 2024Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirementscited by 35
  24. 2024AI Risk Categorization Decoded (AIR 2024): From Government Regulations to Corporate Policiescited by 34
  25. 2024From Distributional to Overton Pluralism: Investigating Large Language Model Alignmentcited by 33
  26. 2024WildFeedback: Aligning LLMs With In-situ User Interactions And Feedbackcited by 31
  27. 2024WordGame: Efficient & Effective LLM Jailbreak via Simultaneous Obfuscation in Query and Responsecited by 28
  28. 2024MRJ-Agent: An Effective Jailbreak Agent for Multi-Round Dialoguecited by 28
  29. 2024Jigsaw Puzzles: Splitting Harmful Questions to Jailbreak Large Language Modelscited by 15
  30. 2025PluralLLM: Pluralistic Alignment in LLMs via Federated Learningcited by 14
  31. 2025Unified Multi-Task Learning & Model Fusion for Efficient Language Model Guardrailingcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Operationalizing Pluralistic Values in Large Language Model Alignment Reveals Trade-offs in Safety, Inclusivity, and Model Behavior
  2. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignment
  3. 2025Unintended Harms of Value-Aligned LLMs: Psychological and Empirical Insights
  4. 2026How Well Can LLM Agents Simulate End-User Security and Privacy Attitudes and Behaviors?
  5. 2025Steerable Pluralism: Pluralistic Alignment via Few-Shot Comparative Regression
  6. 2025Can Finetuing LLMs on Small Human Samples Increase Heterogeneity, Alignment, and Belief-Action Coherence?
  7. 2026PolicyAlign: Direct Policy-Based Safety Alignment for Large Language Models
  8. 2025LLM Can be a Dangerous Persuader: Empirical Study of Persuasion Safety in Large Language Models