mlsanityData to 7 Oct 2026

ExpertPrompting: Instructing Large Language Models to be Distinguished Experts

arXiv 2305.14688 ↗First posted 24 May 2023cs.CL cs.AIcited by 140

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

Abstract (arXiv)

The answering quality of an aligned large language model (LLM) can be drastically improved if treated with proper crafting of prompts. In this paper, we propose ExpertPrompting to elicit the potential of LLMs to answer as distinguished experts. We first utilize In-Context Learning to automatically synthesize detailed and customized descriptions of the expert identity for each specific instruction, and then ask LLMs to provide answer conditioned on such agent background. Based on this augmented prompting strategy, we produce a new set of instruction-following data using GPT-3.5, and train a competitive open-source chat assistant called ExpertLLaMA. We employ GPT4-based evaluation to show that 1) the expert data is of significantly higher quality than vanilla answers, and 2) ExpertLLaMA outperforms existing open-source opponents and achieves 96% of the original ChatGPT's capability. All data and the ExpertLLaMA model will be made publicly available at https://github.com/OFA-Sys/ExpertLLaMA.

Key facts. Cited by 140 papers in the map (data through 7 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 9 of its 17 references that are in the map. First posted on arXiv 24 May 2023.

Builds on

9 of 17 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructions20 Dec
  2. 2022Training language models to follow instructions with human feedback4 Mar

Other references · 7

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  4. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  5. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  6. 2023Instruction Tuning with GPT-4cited by 676
  7. 2023kNN Prompting: Beyond-Context Learning with Calibration-Free Nearest Neighbor Inferencecited by 47

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 25; 2024: 52; 2025: 43; 2026: 20.

140 papers.

Sort:

By area:

  1. 2023Qwen Technical Reportcited by 3,273
  2. 2023Cognitive Architectures for Language Agentscited by 422
  3. 2024Automated Design of Agentic Systemscited by 360
  4. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 276
  5. 2023Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaborationcited by 246
  6. 2023AutoAgents: A Framework for Automatic Agent Generationcited by 211
  7. 2024From Persona to Personalization: A Survey on Role-Playing Language Agentscited by 206
  8. 2023Compositional Chain-of-Thought Prompting for Large Multimodal Modelscited by 160
  9. 2024Meta-Prompting: Enhancing Language Models with Task-Agnostic Scaffoldingcited by 115
  10. 2024EvoAgent: Towards Automatic Multi-Agent Generation via Evolutionary Algorithmscited by 109
  11. 2024Buffer of Thoughts: Thought-Augmented Reasoning with Large Language Modelscited by 91
  12. 2024PersonaGym: Evaluating Persona Agents and LLMscited by 89
38 more
  1. 2023Human Feedback is not Gold Standardcited by 82
  2. 2025LLM Generated Persona is a Promise with a Catchcited by 77
  3. 2023Unleashing the potential of prompt engineering for large language modelscited by 72
  4. 2023Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agentscited by 72
  5. 2023On the Calibration of Large Language Models and Alignmentcited by 72
  6. 2024Self-Contrast: Better Reflection Through Inconsistent Solving Perspectivescited by 71
  7. 2025Audio-CoT: Exploring Chain-of-Thought Reasoning in Large Audio Language Modelcited by 60
  8. 2025Voting or Consensus? Decision-Making in Multi-Agent Debatecited by 55
  9. 2023Do LLMs Possess a Personality? Making the MBTI Test an Amazing Evaluation for Large Language Modelscited by 54
  10. 2023Self-Evolved Diverse Data Sampling for Efficient Instruction Tuningcited by 52
  11. 2024Multimodal Task Vectors Enable Many-Shot Multimodal In-Context Learningcited by 49
  12. 2025EvoFlow: Evolving Diverse Agentic Workflows On The Flycited by 49
  13. 2023Towards LLM-guided Causal Explainability for Black-box Text Classifierscited by 44
  14. 2024Tables as Texts or Images: Evaluating the Table Reasoning Ability of LLMs and MLLMscited by 40
  15. 2023ALYMPICS: LLM Agents Meet Game Theory -- Exploring Strategic Decision-Making with AI Agentscited by 33
  16. 2023Aligning Large Language Models with Human: A Surveycited by 31
  17. 2024CoMM: Collaborative Multi-Agent, Multi-Reasoning-Path Prompting for Complex Problem Solvingcited by 31
  18. 2024Source Code Summarization in the Era of Large Language Modelscited by 30
  19. 2023Bergeron: Combating Adversarial Attacks through a Conscience-Based Alignment Frameworkcited by 29
  20. 2024Debate-to-Write: A Persona-Driven Multi-Agent Framework for Diverse Argument Generationcited by 29
  21. 2024G"odel Agent: A Self-Referential Agent Framework for Recursive Self-Improvementcited by 27
  22. 2024AgentCoord: Visually Exploring Coordination Strategy for LLM-based Multi-Agent Collaborationcited by 26
  23. 2025MAS-ZERO: Designing Multi-Agent Systems with Zero Supervisioncited by 26
  24. 2023Large Language Model Prediction Capabilities: Evidence from a Real-World Forecasting Tournamentcited by 24
  25. 2024Multi-Agent Large Language Models for Conversational Task-Solvingcited by 24
  26. 2025Stay Focused: Problem Drift in Multi-Agent Debatecited by 23
  27. 2024Improving Physics Reasoning in Large Language Models Using Mixture of Refinement Agentscited by 22
  28. 2024AI-Augmented Predictions: LLM Assistants Improve Human Forecasting Accuracycited by 21
  29. 2025Weak-for-Strong: Training Weak Meta-Agent to Harness Strong Executorscited by 20
  30. 2024Identity-Driven Hierarchical Role-Playing Agentscited by 18
  31. 2024Mitigating Hallucination in Fictional Character Role-Playcited by 17
  32. 2025Principled Personas: Defining and Measuring the Intended Effects of Persona Prompting on Task Performancecited by 17
  33. 2023BHASA: A Holistic Southeast Asian Linguistic and Cultural Evaluation Suite for Large Language Modelscited by 16
  34. 2024PersonaFlow: Designing LLM-Simulated Expert Perspectives for Enhanced Research Ideationcited by 16
  35. 2024Enhancing LLM Reasoning with Multi-Path Collaborative Reactive and Reflection agentscited by 15
  36. 2025MALLM: Multi-Agent Large Language Models Frameworkcited by 15
  37. 2024Ask LLMs Directly, "What shapes your bias?": Measuring Social Bias in Large Language Modelscited by 14
  38. 2024Unveiling the Lexical Sensitivity of LLMs: Combinatorial Optimization for Prompt Enhancementcited by 13

50 most cited of 140 All 140 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Chat Vector: A Simple Approach to Equip LLMs with Instruction Following and Model Alignment in New Languages
  2. 2023Ensemble-Instruct: Generating Instruction-Tuning Data with a Heterogeneous Mixture of LMs
  3. 2023Getting MoRE out of Mixture of Language Model Reasoning Experts
  4. 2023Large Language Model Programs
  5. 2024Rewriting Conversational Utterances with Instructed Large Language Models
  6. 2023Ask an Expert: Leveraging Language Models to Improve Strategic Reasoning in Goal-Oriented Dialogue Models
  7. 2023Aurora:Activating Chinese chat capability for Mixtral-8x7B sparse Mixture-of-Experts through Instruction-Tuning
  8. 2024Leveraging Open Knowledge for Advancing Task Expertise in Large Language Models