mlsanityData to 7 Oct 2026

ConvLab-2: An Open-Source Toolkit for Building, Evaluating, and Diagnosing Dialogue Systems

arXiv 2002.04793 ↗First posted 12 Feb 2020cs.CL cs.AIcited by 52

Not on the map. Why?

Abstract (arXiv)

We present ConvLab-2, an open-source toolkit that enables researchers to build task-oriented dialogue systems with state-of-the-art models, perform an end-to-end evaluation, and diagnose the weakness of systems. As the successor of ConvLab (Lee et al., 2019b), ConvLab-2 inherits ConvLab's framework but integrates more powerful dialogue models and supports more datasets. Besides, we have developed an analysis tool and an interactive tool to assist researchers in diagnosing dialogue systems. The analysis tool presents rich statistics and summarizes common mistakes from simulated dialogues, which facilitates error analysis and system improvement. The interactive tool provides a user interface that allows developers to diagnose an assembled dialogue system by interacting with the system and modifying the output of each system component.

Key facts. Cited by 52 papers in the map (data through 7 Oct 2026). First posted on arXiv 12 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 14; 2022: 15; 2023: 11; 2024: 6; 2025: 2; 2026: 4.

52 papers.

Sort:

By area:

  1. 2024ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RLcited by 212
  2. 2021Shades of BLEU, Flavours of Success: The Case of MultiWOZcited by 48
  3. 2021AuGPT: Auxiliary Tasks and Data Augmentation for End-To-End Dialogue with Pre-Trained Language Modelscited by 31
  4. 2024Reliable LLM-based User Simulator for Task-Oriented Dialogue Systemscited by 24
  5. 2021Domain-independent User Simulation with Transformers for Task-oriented Dialogue Systemscited by 23
  6. 2024DuetSim: Building User Simulator with Dual Large Language Models for Task-Oriented Dialoguescited by 23
  7. 2022GenTUS: Simulating User Behaviour and Language in Task-oriented Dialogues with Generative Transformerscited by 22
  8. 2022ConvLab-3: A Flexible Dialogue System Toolkit Based on a Unified Data Formatcited by 22
  9. 2021EmoWOZ: A Large-Scale Corpus and Labelling Scheme for Emotion Recognition in Task-Oriented Dialogue Systemscited by 20
  10. 2023Fantastic Rewards and How to Tame Them: A Case Study on Reward Learning for Task-oriented Dialogue Systemscited by 19
  11. 2023In-Context Learning User Simulators for Task-Oriented Dialog Systemscited by 19
  12. 2023User Simulation with Large Language Models for Evaluating Task-Oriented Dialoguecited by 19
38 more
  1. 2022Dynamic Dialogue Policy for Continual Reinforcement Learningcited by 12
  2. 2023X-RiSAWOZ: High-Quality End-to-End Multilingual Dialogue Datasets and Few-shot Agentscited by 12
  3. 2021Cross-lingual Intermediate Fine-tuning improves Dialogue State Trackingcited by 11
  4. 2022Metaphorical User Simulators for Evaluating Task-oriented Dialogue Systemscited by 9
  5. 2021Uncertainty Measures in Neural Belief Tracking and the Effects on Dialogue Policy Performancecited by 8
  6. 2021DEUS: A Data-driven Approach to Estimate User Satisfaction in Multi-turn Dialoguescited by 7
  7. 2021End-to-End Conversational Search for Online Shopping with Utterance Transfercited by 7
  8. 2022DialCrowd 2.0: A Quality-Focused Dialog System Crowdsourcing Toolkitcited by 6
  9. 2022A Generative User Simulator with GPT-based Architecture and Goal State Tracking for Reinforced Multi-Domain Dialog Systemscited by 6
  10. 2024Enhancing End-to-End Multi-Task Dialogue Systems: A Study on Intrinsic Motivation Reinforcement Learning Algorithms for Improved Training and Adaptabilitycited by 6
  11. 2021Integrating Pretrained Language Model for Dialogue Policy Learningcited by 5
  12. 2022Post-processing Networks: Method for Optimizing Pipeline Task-oriented Dialogue Systems using Reinforcement Learningcited by 5
  13. 2022Dialogue Term Extraction using Transfer Learning and Topological Data Analysiscited by 5
  14. 2025clem:todd: A Framework for the Systematic Benchmarking of LLM-Based Task-Oriented Dialogue System Realisationscited by 4
  15. 2021A Hybrid Task-Oriented Dialog System with Domain and Task Adaptive Pretrainingcited by 3
  16. 2021High-Quality Diversification for Task-Oriented Dialogue Systemscited by 3
  17. 2022"Think Before You Speak": Improving Multi-Action Dialog Policy by Planning Single-Action Dialogscited by 3
  18. 2022Dialogue Evaluation with Offline Reinforcement Learningcited by 3
  19. 2022An Exploratory Study of AI System Risk Assessment from the Lens of Data Distribution and Uncertaintycited by 3
  20. 2025MATRIX: Multi-Agent simulaTion fRamework for safe Interactions and conteXtual clinical conversational evaluationcited by 3
  21. 2021A Unified Pre-training Framework for Conversational AIcited by 2
  22. 2021How to Evaluate Your Dialogue Models: A Review of Approachescited by 2
  23. 2022Adaptive Natural Language Generation for Task-oriented Dialogue via Reinforcement Learningcited by 2
  24. 2022Graph Neural Network Policies and Imitation Learning for Multi-Domain Task-Oriented Dialoguescited by 2
  25. 2021Joint System-Wise Optimization for Pipeline Goal-Oriented Dialog Systemcited by 1
  26. 2022BotSIM: An End-to-End Bot Simulation Framework for Commercial Task-Oriented Dialog Systemscited by 1
  27. 2023Few-Shot Structured Policy Learning for Multi-Domain and Multi-Task Dialoguescited by 1
  28. 2023Dialog-to-Actions: Building Task-Oriented Dialogue System via Action-Level Generationcited by 1
  29. 2024DIALIGHT: Lightweight Multilingual Development and Evaluation of Task-Oriented Dialogue Systems with Large Language Modelscited by 1
  30. 2022BotSIM: An End-to-End Bot Simulation Toolkit for Commercial Task-Oriented Dialog Systemscited by 0
  31. 2023A Personalized Utterance Style (PUS) based Dialogue Strategy for Efficient Service Requirement Elicitationcited by 0
  32. 2023Why Guided Dialog Policy Learning performs well? Understanding the role of adversarial learning and its alternativecited by 0
  33. 2023JoTR: A Joint Transformer and Reinforcement Learning Framework for Dialog Policy Learningcited by 0
  34. 2023Dialog Action-Aware Transformer for Dialog Policy Learningcited by 0
  35. 2023Evaluating Robustness of Dialogue Summarization Models in the Presence of Naturally Occurring Variationscited by 0
  36. 2024WEBDial, a Multi-domain, Multitask Statistical Dialogue Framework with RDFcited by 0
  37. 2026Orcheo: A Modular Full-Stack Platform for Conversational Searchcited by 0
  38. 2026Bridging Reasoning and Action: Hybrid LLM-RL Framework for Efficient Cross-Domain Task-Oriented Dialoguecited by 0

50 most cited of 52 All 52 (JSON)