mlsanityData to 7 Oct 2026

GEM: Graph-Enhanced Mixture-of-Experts with ReAct Agents for Dialogue State Tracking

arXiv 2605.04449 ↗First posted 6 May 2026cs.CL cs.AIcited by 0

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

also in 1 more
Abstract (arXiv)

Dialogue State Tracking (DST) requires precise extraction of structured information from multi-domain conversations, a task where Large Language Models (LLMs) struggle despite their impressive general capabilities. We present GEM (Graph-Enhanced Mixture-of-Experts), a novel framework that combines language models and graph-structured dialogue understanding with ReAct agent-based reasoning for superior DST performance. Our approach dynamically routes between specialized experts: a Graph Neural Network that captures dialogue structure and turn-level dependencies, and a finetuned T5-Small encoder-decoder for sequence modeling, coordinated by an intelligent router. For complex value generation tasks, we integrate ReAct agents that perform structured reasoning over dialogue context. On MultiWOZ 2.2, GEM achieves 65.19% Joint Goal Accuracy, substantially outperforming end-to-end LLM approaches (best: 38.43%) and surpassing state-of-the-art (SOTA) methods including TOATOD (63.79%), D3ST (58.70%), and Diable (56.48%). Our graph-enhanced mixture-of-experts architecture with ReAct integration demonstrates that combining structured dialogue representation with dynamic expert routing and agent-based reasoning provides a powerful paradigm for dialogue state tracking, achieving superior accuracy while maintaining computational efficiency through selective expert activation.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 28 of its 48 references that are in the map. First posted on arXiv 6 May 2026.

Builds on

28 of 48 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Large Language Models as Zero-shot Dialogue State Tracker through Function Calling16 Feb

Other references · 27

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2017Graph Attention Networkscited by 8,854
  5. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  6. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  7. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  8. 2021How Attentive are Graph Attention Networks?cited by 818
  9. 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 760
  10. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 663
  11. 2023C-Pack: Packed Resources For General Chinese Embeddingscited by 556
  12. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 554
15 more
  1. 2019Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systemscited by 215
  2. 2024OpenMoE: An Early Effort on Open Mixture-of-Experts Language Modelscited by 185
  3. 2020TripPy: A Triple Copy Strategy for Value Independent Neural Dialog State Trackingcited by 130
  4. 2024GNN-RAG: Graph Neural Retrieval for Large Language Model Reasoningcited by 128
  5. 2022UniKGQA: Unified Retrieval and Reasoning for Solving Multi-hop Question Answering Over Knowledge Graphcited by 100
  6. 2023Knowledge-Driven CoT: Exploring Faithful Reasoning in LLMs for Knowledge-intensive Question Answeringcited by 97
  7. 2021RnG-KBQA: Generation Augmented Iterative Ranking for Knowledge Base Question Answeringcited by 86
  8. 2019Find or Classify? Dual Strategy for Slot-Value Predictions on Multi-Domain Dialog State Trackingcited by 75
  9. 2024A Survey on Inference Optimization Techniques for Mixture of Experts Modelscited by 46
  10. 2020A Sequence-to-Sequence Approach to Dialogue State Trackingcited by 43
  11. 2023Task-Optimized Adapters for an End-to-End Task-Oriented Dialogue Systemcited by 30
  12. 2023Faithful Question Answering with Monte-Carlo Planningcited by 16
  13. 2023InstructTODS: Large Language Models for End-to-End Task-Oriented Dialogue Systemscited by 5
  14. 2023Diable: Efficient Dialogue State Tracking as Operations on Tablescited by 4
  15. 2024Inference is All You Need: Self Example Retriever for Cross-domain Dialogue State Tracking with ChatGPTcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Divide, Conquer, and Combine: Mixture of Semantic-Independent Experts for Zero-Shot Dialogue State Tracking
  2. 2026Learning to Reason and Use Tools through Unsupervised Fine-Tuning in Task-Oriented Dialog Systems
  3. 2022Towards Large-Scale Interpretable Knowledge Graph Reasoning for Dialogue Systems
  4. 2026Context-Agent: Dynamic Discourse Trees for Non-Linear Dialogue
  5. 2024Enhancing Dialogue State Tracking Models through LLM-backed User-Agents Simulation
  6. 2024Benchmarks Underestimate the Readiness of Multi-lingual Dialogue Agents
  7. 2024Many Hands Make Light Work: Task-Oriented Dialogue System with Module-Based Mixture-of-Experts
  8. 2024Chain of Thought Explanation for Dialogue State Tracking