mlsanityData to 14 Oct 2026

Extracting Persona Subspaces Through Iterative Nullspace Projection For Modulation

arXiv 2610.04676 ↗First posted 3 Oct 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) can adopt distinct personas to tune their semantics, expertise, and perspective to different users and tasks. Precise control over these traits is critical to ensure safety and reliability in model behavior. Existing methods like activation steering and prompt-based persona induction reduce a persona to a single dominant direction, missing the finer, nested traits that emerge only once that dominant signal is factored out. We introduce modulation as a setting where the persona context is already embedded in the content being manipulated, requiring control methods to amplify or suppress a trait already present rather than inject it from scratch. PaSS is an inference-time control paradigm that models personas as multi-dimensional subspaces in a model's latent space without supervised contrastive examples. The persona subspaces are extracted via iterative concept erasure and applied to modulate persona-guided generation without retraining. To extract this subspace, we use Iterative Nullspace Projections (INLP) to linearly and iteratively isolate persona-specific directions. We causally evaluate six personas against diverse tasks like MATH-500, TinyAlpaca, GSM8K, and IFEval, showing that discriminative, iterative subspace extraction captures diverse traits underlying a given persona, enabling stronger and larger modulation than single-direction additive methods, while maintaining content fidelity. We further study individual peeled directions within each subspace to uncover the distinct aspects of persona behavior they encode. Overall, we show that persona subspaces offer a controllable, interpretable, and generalizable framework for modulating LLM behavior without sacrificing task performance.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 80 of its 89 references that are in the map. First posted on arXiv 3 Oct 2026.

Builds on

80 of 89 references.

References · 80

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2023Let's Verify Step by Stepcited by 3,789
  3. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  4. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  5. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,228
  6. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  7. 2022Toy Models of Superpositioncited by 971
  8. 2023Steering Language Models With Activation Engineeringcited by 922
  9. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 767
  10. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  11. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 701
  12. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 623
38 more
  1. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 536
  2. 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 488
  3. 2020Null It Out: Guarding Protected Attributes by Iterative Nullspace Projectioncited by 455
  4. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 435
  5. 2025LLMs Get Lost In Multi-Turn Conversationcited by 429
  6. 2023JudgeLM: Fine-tuned Large Language Models are Scalable Judgescited by 367
  7. 2025Persona Vectors: Monitoring and Controlling Character Traits in Language Modelscited by 345
  8. 2023Language Models Represent Space and Timecited by 316
  9. 2024tinyBenchmarks: evaluating LLMs with fewer examplescited by 310
  10. 2024Knowledge Conflicts for LLMs: A Surveycited by 283
  11. 2025AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoderscited by 218
  12. 2024Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalizationcited by 208
  13. 2023Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Modelscited by 207
  14. 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretabilitycited by 205
  15. 2023Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMscited by 199
  16. 2022On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoningcited by 194
  17. 2026Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnetcited by 191
  18. 2024Analyzing the Generalization and Reliability of Steering Vectorscited by 143
  19. 2024Personalization of Large Language Models: A Surveycited by 104
  20. 2024PersonaGym: Evaluating Persona Agents and LLMscited by 97
  21. 2023Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMscited by 91
  22. 2024Attention Heads of Large Language Models: A Surveycited by 67
  23. 2023CharacterChat: Learning towards Conversational AI with Personalized Social Supportcited by 60
  24. 2023Do LLMs Possess a Personality? Making the MBTI Test an Amazing Evaluation for Large Language Modelscited by 56
  25. 2025Angular Steering: Behavior Control via Rotation in Activation Spacecited by 53
  26. 2022Kernelized Concept Erasurecited by 48
  27. 2024From Tokens to Words: On the Inner Lexicon of LLMscited by 48
  28. 2023Personas as a Way to Model Truthfulness in Language Modelscited by 47
  29. 2024Thinking Fair and Slow: On the Efficacy of Structured Prompts for Debiasing Language Modelscited by 45
  30. 2025The Prompt Makes the Person(a): A Systematic Evaluation of Sociodemographic Persona Prompting for Large Language Modelscited by 42
  31. 2024Prompting Techniques for Reducing Social Bias in LLMs through System 1 and System 2 Cognitive Processescited by 39
  32. 2025Sycophancy Is Not One Thing: Causal Separation of Sycophantic Behaviors in LLMscited by 37
  33. 2024Emergence of Hidden Capabilities: Exploring Learning Dynamics in Concept Spacecited by 35
  34. 2019Generating Persona Consistent Dialogues by Exploiting Natural Language Inferencecited by 34
  35. 2025From 1,000,000 Users to Every User: Scaling Up Personalized Preference for User-level Alignmentcited by 30
  36. 2024PICLe: Eliciting Diverse Behaviors from Large Language Models with Persona In-Context Learningcited by 29
  37. 2025SynthesizeMe! Inducing Persona-Guided Prompts for Personalized Reward Models in LLMscited by 29
  38. 2025A Survey on Personalized and Pluralistic Preference Alignment in Large Language Modelscited by 27

50 most cited of 80 All 80 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026The Persona Hierarchy Model: Understanding Contextual Generalization in Fine-Tuning LLMs
  2. 2026Your Language Model Secretly Contains Personality Subnetworks
  3. 2026The Assistant Axis: Situating and Stabilizing the Default Persona of Language Models
  4. 2025PILOT: Steering Synthetic Data Generation with Psychological & Linguistic Output Targeting
  5. 2026K/V-Cache Interventions Dissociate Representation Alignment from Persona Expression in Decoder-Only Language Models
  6. 2025Dissecting Persona-Driven Reasoning in Language Models via Activation Patching
  7. 2026Facet-Level Persona Control by Trait-Activated Routing with Contrastive SAE for Role-Playing LLMs
  8. 2025The Geometry of Persona: Disentangling Personality from Reasoning in Large Language Models