# Dialogue Term Extraction using Transfer Learning and Topological Data Analysis

arXiv: https://arxiv.org/abs/2208.10448 · HTML: https://mlsanity.com/paper/2208.10448/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 89 references that are in the map.
- First posted on arXiv 22 Aug 2022.

## Builds on (most cited first, 26 of 26)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [LaMDA: Language Models for Dialog Applications](https://mlsanity.com/paper/2201.08239/) (2022)
- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling](https://mlsanity.com/paper/1810.00278/) (2018)
- [Towards a Human-like Open-Domain Chatbot](https://mlsanity.com/paper/2001.09977/) (2020)
- [Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Dataset](https://mlsanity.com/paper/1909.05855/) (2019)
- [Learning Word Vectors for 157 Languages](https://mlsanity.com/paper/1802.06893/) (2018)
- [Persistence Images: A Stable Vector Representation of Persistent Homology](https://mlsanity.com/paper/1507.06217/) (2015)
- [TripPy: A Triple Copy Strategy for Value Independent Neural Dialog State Tracking](https://mlsanity.com/paper/2005.02877/) (2020)
- [Ripser: efficient computation of Vietoris-Rips persistence barcodes](https://mlsanity.com/paper/1908.02518/) (2019)
- [GALAXY: A Generative Pre-trained Model for Task-Oriented Dialog with Semi-Supervised Learning and Explicit Policy Injection](https://mlsanity.com/paper/2111.14592/) (2021)
- [Dialogue State Tracking with a Language Model using Schema-Driven Prompting](https://mlsanity.com/paper/2109.07506/) (2021)
- [ConvLab-2: An Open-Source Toolkit for Building, Evaluating, and Diagnosing Dialogue Systems](https://mlsanity.com/paper/2002.04793/) (2020)
- [Overview of the Ninth Dialog System Technology Challenge: DSTC9](https://mlsanity.com/paper/2011.06486/) (2020)
- [Artificial Text Detection via Examining the Topology of Attention Maps](https://mlsanity.com/paper/2109.04825/) (2021)
- [AuGPT: Auxiliary Tasks and Data Augmentation for End-To-End Dialogue with Pre-Trained Language Models](https://mlsanity.com/paper/2102.05126/) (2021)
- [Advanced Semantics for Commonsense Knowledge Extraction](https://mlsanity.com/paper/2011.00905/) (2020)
- [Persformer: A Transformer Architecture for Topological Machine Learning](https://mlsanity.com/paper/2112.15210/) (2021)
- [Topology of Word Embeddings: Singularities Reflect Polysemy](https://mlsanity.com/paper/2011.09413/) (2020)
- [Unsupervised Slot Schema Induction for Task-oriented Dialog](https://mlsanity.com/paper/2205.04515/) (2022)
- [Knowing What You Know: Calibrating Dialogue Belief State Distributions via Ensembles](https://mlsanity.com/paper/2010.02586/) (2020)
- [Structure Extraction in Task-Oriented Dialogues with Slot Clustering](https://mlsanity.com/paper/2203.00073/) (2022)
- [TiFi: Taxonomy Induction for Fictional Domains [Extended version]](https://mlsanity.com/paper/1901.10263/) (2019)

## Built on by (most cited first, 5 of 5)

- [TOPFORMER: Topology-Aware Authorship Attribution of Deepfake Texts with Diverse Writing Styles](https://mlsanity.com/paper/2309.12934/) (2023)
- [Topological Data Analysis for Speech Processing](https://mlsanity.com/paper/2211.17223/) (2022)
- [Local Topology Measures of Contextual Language Model Latent Spaces With Applications to Dialogue Term Extraction](https://mlsanity.com/paper/2408.03706/) (2024)
- [Topological Data Analysis Applications in Natural Language Processing: A Survey](https://mlsanity.com/paper/2411.10298/) (2024)
- [Text-to-SQL Task-oriented Dialogue Ontology Construction](https://mlsanity.com/paper/2507.23358/) (2025)

## Abstract (arXiv)

Goal oriented dialogue systems were originally designed as a natural language interface to a fixed data-set of entities that users might inquire about, further described by domain, slots, and values. As we move towards adaptable dialogue systems where knowledge about domains, slots, and values may change, there is an increasing need to automatically extract these terms from raw dialogues or related non-dialogue data on a large scale. In this paper, we take an important step in this direction by exploring different features that can enable systems to discover realizations of domains, slots, and values in dialogues in a purely data-driven fashion. The features that we examine stem from word embeddings, language modelling features, as well as topological features of the word embedding space. To examine the utility of each feature set, we train a seed model based on the widely used MultiWOZ data-set. Then, we apply this model to a different corpus, the Schema-Guided Dialogue data-set. Our method outperforms the previously proposed approach that relies solely on word embeddings. We also demonstrate that each of the features is responsible for discovering different kinds of content. We believe our results warrant further research towards ontology induction, and continued harnessing of topological data analysis for dialogue and natural language processing research.