mlsanityData to 14 Oct 2026

FOCUS: Decoupling Expert Personas in LLMs to Enhance Domain Expert Capabilities

arXiv 2608.05611 ↗First posted 6 Aug 2026cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) can exhibit diverse personas, and activating expert personas has been shown to improve domain expertise and task accuracy. However, existing persona control methods often suffer from cross-domain coupling, which may lead to overly aggressive behavior in high-caution domains such as healthcare, or excessive conservatism in risk-sensitive domains such as financial trading. To address this issue, we propose FOCUS (textbfunderlineFine-tuning with textbfunderlineOrthogonal textbfunderlineControl for textbfunderlineUncoupled personatextbfunderlineS). FOCUS first automatically extracts expert persona vectors from LLMs, then applies orthogonal decomposition to decouple domain-specific expert personas, and finally introduces an expert gating module to adaptively control persona activation according to task contexts. With a two-stage training strategy and a gated selection regularizer, the model learns to activate appropriate personas for both single-domain and cross-domain tasks. Experiments on financial, legal, medical, and cross-domain benchmarks show that FOCUS improves task accuracy and outperforms existing persona control methods. Our code is available at hrefhttps://anonymous.4open.science/r/openpersona-48F4this url.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 32 of its 41 references that are in the map. First posted on arXiv 6 Aug 2026.

Builds on

32 of 41 references.

References · 32

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  3. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  4. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  5. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  6. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  7. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
  8. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  9. 2022Toy Models of Superpositioncited by 971
  10. 2023Steering Language Models With Activation Engineeringcited by 922
  11. 2023Large Language Models as Optimizerscited by 864
  12. 2020Null It Out: Guarding Protected Attributes by Iterative Nullspace Projectioncited by 455
20 more
  1. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 359
  2. 2025Persona Vectors: Monitoring and Controlling Character Traits in Language Modelscited by 345
  3. 2023LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Modelscited by 336
  4. 2024BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domainscited by 301
  5. 2023LEACE: Perfect linear concept erasure in closed formcited by 267
  6. 2021CUAD: An Expert-Annotated NLP Dataset for Legal Contract Reviewcited by 236
  7. 2023PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traitscited by 234
  8. 2024Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalizationcited by 208
  9. 2024TradingAgents: Multi-Agents LLM Financial Trading Frameworkcited by 188
  10. 2024UltraMedical: Building Specialized Generalists in Biomedicinecited by 67
  11. 2025Steering Out-of-Distribution Generalization with Concept Ablation Fine-Tuningcited by 39
  12. 2024Learning to Generate Explainable Stock Predictions using Self-Reflective Large Language Modelscited by 36
  13. 2025Qwen Look Again: Guiding Vision-Language Reasoning Models to Re-attention Visual Informationcited by 28
  14. 2025Representation Engineering for Large-Language Models: Survey and Research Challengescited by 21
  15. 2023Making Harmful Behaviors Unlearnable for Large Language Modelscited by 20
  16. 2025Domaino1s: Guiding LLM Reasoning for Explainable Answers in High-Stakes Domainscited by 11
  17. 2025MedEthicsQA: A Comprehensive Question Answering Benchmark for Medical Ethics Evaluation of LLMscited by 8
  18. 2025GraphSOS: Graph Sampling and Order Selection to Help LLMs Understand Graphs Bettercited by 4
  19. 2025Personas within Parameters: Fine-Tuning Small Language Models with Low-Rank Adapters to Mimic User Behaviorscited by 4
  20. 2026Towards Order Fairness: Mitigating LLMs Order Sensitivity through Dual Group Advantage Optimizationcited by 1

Built on by

Not cited yet.