# GLM-Dialog: Noise-tolerant Pre-training for Knowledge-grounded Dialogue Generation arXiv: https://arxiv.org/abs/2302.14401 ยท HTML: https://mlsanity.com/paper/2302.14401/ ## Key facts - Cited by 9 papers in the map (data through 14 Oct 2026). - Builds on 30 of its 41 references that are in the map. - First posted on arXiv 28 Feb 2023. ## Builds on (most cited first, 30 of 30) - [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019) - [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022) - [LaMDA: Language Models for Dialog Applications](https://mlsanity.com/paper/2201.08239/) (2022) - [GLM: General Language Model Pretraining with Autoregressive Blank Infilling](https://mlsanity.com/paper/2103.10360/) (2021) - [GLM-130B: An Open Bilingual Pre-trained Model](https://mlsanity.com/paper/2210.02414/) (2022) - [Personalizing Dialogue Agents: I have a dog, do you have pets too?](https://mlsanity.com/paper/1801.07243/) (2018) - [Unified Language Model Pre-training for Natural Language Understanding and Generation](https://mlsanity.com/paper/1905.03197/) (2019) - [Wizard of Wikipedia: Knowledge-Powered Conversational agents](https://mlsanity.com/paper/1811.01241/) (2018) - [Revisiting Pre-Trained Models for Chinese Natural Language Processing](https://mlsanity.com/paper/2004.13922/) (2020) - [BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage](https://mlsanity.com/paper/2208.03188/) (2022) - [A Knowledge-Grounded Neural Conversation Model](https://mlsanity.com/paper/1702.01932/) (2017) - [DuReader: a Chinese Machine Reading Comprehension Dataset from Real-world Applications](https://mlsanity.com/paper/1711.05073/) (2017) - [Knowledge-Grounded Dialogue Generation with Pre-trained Language Models](https://mlsanity.com/paper/2010.08824/) (2020) - [Proactive Human-Machine Conversation with Explicit Conversation Goals](https://mlsanity.com/paper/1906.05572/) (2019) - [A Large-Scale Chinese Short-Text Conversation Dataset](https://mlsanity.com/paper/2008.03946/) (2020) - [What Changes Can Large-scale Language Models Bring? Intensive Study on HyperCLOVA: Billions-scale Korean Generative Pretrained Transformers](https://mlsanity.com/paper/2109.04650/) (2021) - [DialogLM: Pre-trained Model for Long Dialogue Understanding and Summarization](https://mlsanity.com/paper/2109.02492/) (2021) - [KdConv: A Chinese Multi-domain Dialogue Dataset Towards Multi-turn Knowledge-driven Conversation](https://mlsanity.com/paper/2004.04100/) (2020) - [DRCD: a Chinese Machine Reading Comprehension Dataset](https://mlsanity.com/paper/1806.00920/) (2018) - [GODEL: Large-Scale Pre-Training for Goal-Directed Dialog](https://mlsanity.com/paper/2206.11309/) (2022) - [PLATO-XL: Exploring the Large-scale Pre-training of Dialogue Generation](https://mlsanity.com/paper/2109.09519/) (2021) - [Dataset and Neural Recurrent Sequence Labeling Model for Open-Domain Factoid Question Answering](https://mlsanity.com/paper/1607.06275/) (2016) - [NaturalConv: A Chinese Dialogue Dataset Towards Multi-turn Topic-driven Conversation](https://mlsanity.com/paper/2103.02548/) (2021) - [Controlling Conditional Language Models without Catastrophic Forgetting](https://mlsanity.com/paper/2112.00791/) (2021) - [EVA2.0: Investigating Open-Domain Chinese Dialogue Systems with Large-Scale Pre-Training](https://mlsanity.com/paper/2203.09313/) (2022) - [Towards Boosting the Open-Domain Chatbot with Human Feedback](https://mlsanity.com/paper/2208.14165/) (2022) - [Medical Dialogue Response Generation with Pivotal Information Recalling](https://mlsanity.com/paper/2206.08611/) (2022) - [Conversational agents for fostering curiosity-driven learning in children](https://mlsanity.com/paper/2204.03546/) (2022) - [Link the World: Improving Open-domain Conversation with Dynamic Spatiotemporal-aware Knowledge](https://mlsanity.com/paper/2206.14000/) (2022) - [PLATO-K: Internal and External Knowledge Enhanced Dialogue Generation](https://mlsanity.com/paper/2211.00910/) (2022) ## Built on by (most cited first, 9 of 9) - [Interactive Agents: Simulating Counselor-Client Psychological Counseling via Role-Playing LLM-to-LLM Interactions](https://mlsanity.com/paper/2408.15787/) (2024) - [PerLTQA: A Personal Long-Term Memory Dataset for Memory Classification, Retrieval, and Synthesis in Question Answering](https://mlsanity.com/paper/2402.16288/) (2024) - [HSplitLoRA: A Heterogeneous Split Parameter-Efficient Fine-Tuning Framework for Large Language Models](https://mlsanity.com/paper/2505.02795/) (2025) - [A Survey of the Evolution of Language Model-Based Dialogue Systems: Data, Task and Models](https://mlsanity.com/paper/2311.16789/) (2023) - [Chain of Thought Prompting Elicits Knowledge Augmentation](https://mlsanity.com/paper/2307.01640/) (2023) - [Think Before You Speak: Cultivating Communication Skills of Large Language Models via Inner Monologue](https://mlsanity.com/paper/2311.07445/) (2023) - [Mining Intrinsic Rewards from LLM Hidden States for Efficient Best-of-N Sampling](https://mlsanity.com/paper/2505.12225/) (2025) - [SoAy: A Solution-based LLM API-using Methodology for Academic Information Seeking](https://mlsanity.com/paper/2405.15165/) (2024) - [Untangle the KNOT: Interweaving Conflicting Knowledge and Reasoning Skills in Large Language Models](https://mlsanity.com/paper/2404.03577/) (2024) ## Abstract (arXiv) We present GLM-Dialog, a large-scale language model (LLM) with 10B parameters capable of knowledge-grounded conversation in Chinese using a search engine to access the Internet knowledge. GLM-Dialog offers a series of applicable techniques for exploiting various external knowledge including both helpful and noisy knowledge, enabling the creation of robust knowledge-grounded dialogue LLMs with limited proper datasets. To evaluate the GLM-Dialog more fairly, we also propose a novel evaluation method to allow humans to converse with multiple deployed bots simultaneously and compare their performance implicitly instead of explicitly rating using multidimensional metrics.Comprehensive evaluations from automatic to human perspective demonstrate the advantages of GLM-Dialog comparing with existing open source Chinese dialogue models. We release both the model checkpoint and source code, and also deploy it as a WeChat application to interact with users. We offer our evaluation platform online in an effort to prompt the development of open source models and reliable dialogue evaluation systems. The additional easy-to-use toolkit that consists of short text entity linking, query generation, and helpful knowledge classification is also released to enable diverse applications. All the source code is available on Github.