mlsanityData to 7 Oct 2026

Ask an Expert: Leveraging Language Models to Improve Strategic Reasoning in Goal-Oriented Dialogue Models

arXiv 2305.17878 ↗First posted 29 May 2023cs.CL cs.AI cs.HCcited by 30

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Existing dialogue models may encounter scenarios which are not well-represented in the training data, and as a result generate responses that are unnatural, inappropriate, or unhelpful. We propose the "Ask an Expert" framework in which the model is trained with access to an "expert" which it can consult at each turn. Advice is solicited via a structured dialogue with the expert, and the model is optimized to selectively utilize (or ignore) it given the context and dialogue history. In this work the expert takes the form of an LLM. We evaluate this framework in a mental health support domain, where the structure of the expert conversation is outlined by pre-specified prompts which reflect a reasoning strategy taught to practitioners in the field. Blenderbot models utilizing "Ask an Expert" show quality improvements across all expert sizes, including those with fewer parameters than the dialogue model itself. Our best model provides a ~ 10% improvement over baselines, approaching human-level scores on "engingingness" and "helpfulness" metrics.

Key facts. Cited by 30 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 39 of its 66 references that are in the map. First posted on arXiv 29 May 2023.

Builds on

39 of 66 references.

References · 39

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  4. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  5. 2017Deep reinforcement learning from human preferencescited by 3,846
  6. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  7. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  8. 2019Language Models as Knowledge Bases?cited by 1,922
  9. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  10. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  11. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,068
  12. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
27 more
  1. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  2. 2021BARTScore: Evaluating Generated Text as Text Generationcited by 719
  3. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 696
  4. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 668
  5. 2017DailyDialog: A Manually Labelled Multi-turn Dialogue Datasetcited by 649
  6. 2020Recipes for building an open-domain chatbotcited by 593
  7. 2020Towards a Human-like Open-Domain Chatbotcited by 467
  8. 2021Towards Emotional Support Dialog Systemscited by 294
  9. 2021Symbolic Knowledge Distillation: from General Language Models to Commonsense Modelscited by 270
  10. 2021Beyond Goldfish Memory: Long-Term Open-Domain Conversationcited by 266
  11. 2021Internet-Augmented Dialogue Generationcited by 217
  12. 2022BlenderBot 3: a deployed conversational agent that continually learns to responsibly engagecited by 197
  13. 2021Generating Datasets with Pretrained Language Modelscited by 192
  14. 2021Reframing Instructional Prompts to GPTk's Languagecited by 176
  15. 2017ParlAI: A Dialog Research Software Platformcited by 139
  16. 2017Key-Value Retrieval Networks for Task-Oriented Dialoguecited by 137
  17. 2020Can You Put it All Together: Evaluating Conversational Agents' Ability to Blend Skillscited by 130
  18. 2020Human-centric Dialog Training via Offline Reinforcement Learningcited by 92
  19. 2019ACUTE-EVAL: Improved Dialogue Evaluation with Optimized Questions and Multi-turn Comparisonscited by 83
  20. 2020PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum Learningcited by 79
  21. 2021Towards Zero-Label Language Learningcited by 76
  22. 2018Mem2Seq: Effectively Incorporating Knowledge Bases into End-to-End Task-Oriented Dialog Systemscited by 70
  23. 2021Knowledge Inheritance for Pre-trained Language Modelscited by 38
  24. 2021SaFeRDialogues: Taking Feedback Gracefully after Conversational Safety Failurescited by 37
  25. 2021Improving Biomedical Pretrained Language Models with Knowledgecited by 36
  26. 2019Know More about Each Other: Evolving Dialogue Strategy via Compound Assessmentcited by 8
  27. 2022Converse: A Tree-Based Modular Task-Oriented Dialogue Systemcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 9; 2025: 11; 2026: 7.

30 papers.

Sort:

By area:

  1. 2023Plug-and-Play Policy Planner for Large Language Model Powered Dialogue Agentscited by 76
  2. 2024AGILE: A Novel Reinforcement Learning Framework of LLM Agentscited by 45
  3. 2024Large Language Models in Mental Health Care: a Scoping Reviewcited by 34
  4. 2024Planning Like Human: A Dual-process Framework for Dialogue Planningcited by 32
  5. 2024Data Augmentation using Large Language Models: Data Perspectives, Learning Paradigms and Challengescited by 30
  6. 2023Task-Adaptive Tokenization: Enhancing Long-Form Text Generation Efficacy in Mental Health and Beyondcited by 22
  7. 2024Strength Lies in Differences! Improving Strategy Planning for Non-collaborative Dialogues via Diversified User Simulationcited by 21
  8. 2025PRINCIPLES: Synthetic Strategy Memory for Proactive Dialogue Agentscited by 15
  9. 2024Simulation-Free Hierarchical Latent Policy Planning for Proactive Dialoguescited by 14
  10. 2025TituLLMs: A Family of Bangla LLMs with Comprehensive Benchmarkingcited by 9
  11. 2025Dialogue Systems for Emotional Support via Value Reinforcementcited by 8
  12. 2024Script-Based Dialog Policy Planning for LLM-Powered Conversational Agents: A Basic Architecture for an "AI Therapist"cited by 7
18 more
  1. 2025Simulating Before Planning: Constructing Intrinsic User World Model for User-Tailored Dialogue Policy Planningcited by 7
  2. 2025MIND: Towards Immersive Psychological Healing with Multi-agent Inner Dialoguecited by 6
  3. 2024"I Like Sunnie More Than I Expected!": Exploring User Expectation and Perception of an Anthropomorphic LLM-based Conversational Agent for Well-Being Supportcited by 5
  4. 2025Towards Open-Ended Emotional Support Conversations in LLMs via Reinforcement Learning with Future-Oriented Rewardscited by 5
  5. 2025DialogXpert: Driving Intelligent and Emotion-Aware Conversations through Online Value-Based Reinforcement Learning with LLM Priorscited by 4
  6. 2025EmoFSM: A Finite State Machine for Emotional Support Conversationcited by 3
  7. 2023DiagGPT: An LLM-based and Multi-agent Dialogue System with Automatic Topic Management for Flexible Task-Oriented Dialoguecited by 2
  8. 2024SAGA: A Participant-specific Examination of Story Alternatives and Goal Applicability for a Deeper Understanding of Complex Eventscited by 2
  9. 2025Somatic in the East, Psychological in the West?: Investigating Clinically-Grounded Cross-Cultural Depression Symptom Expression in LLMscited by 2
  10. 2026Responsible Evaluation of AI for Mental Healthcited by 2
  11. 2025A General Highly Accurate Online Planning Method Integrating Large Language Models into Nested Rollout Policy Adaptation for Dialogue Taskscited by 1
  12. 2025Planning with Diffusion Models for Target-Oriented Dialogue Systemscited by 0
  13. 2026The Art of Socratic Inquiry: A Framework for Proactive Template-Guided Therapeutic Conversation Generationcited by 0
  14. 2026EmoLLM: Appraisal-Grounded Cognitive-Emotional Co-Reasoning in Large Language Modelscited by 0
  15. 2026PRISMA: Preference-Reinforced Self-Training Approach for Interpretable Emotionally Intelligent Negotiation Dialoguescited by 0
  16. 2026UP-NRPA: User Portrait based Nested Rollout Policy Adaptation for Planning with Large Language Models in Goal-oriented Dialogue Systemscited by 0
  17. 2026Everyone is unique: Towards Behaviorally Heterogeneous Negotiation Dialogue Systems for Debt Collectioncited by 0
  18. 2026Proactive Dialogue Policy Optimization via Cognitive-State Transitioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Grounded in Reality: Learning and Deploying Proactive LLM from Offline Logs
  2. 2022A Mixture-of-Expert Approach to RL-based Dialogue Management
  3. 2023ExpertPrompting: Instructing Large Language Models to be Distinguished Experts
  4. 2023Prompting and Evaluating Large Language Models for Proactive Dialogues: Clarification, Target-guided, and Non-collaboration
  5. 2025ESC-Judge: A Framework for Comparing Emotional Support Conversational Agents
  6. 2023DERA: Enhancing Large Language Model Completions with Dialog-Enabled Resolving Agents
  7. 2025MindEval: Benchmarking Language Models on Multi-turn Mental Health Support
  8. 2026MoBayes: A Modular Bayesian Framework for Separating Reasoning from Language in Conversational Clinical Decision Support