# Investigating Robustness of Dialog Models to Popular Figurative Language Constructs

arXiv: https://arxiv.org/abs/2110.00687 · HTML: https://mlsanity.com/paper/2110.00687/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 39 references that are in the map.
- First posted on arXiv 1 Oct 2021.

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Personalizing Dialogue Agents: I have a dog, do you have pets too?](https://mlsanity.com/paper/1801.07243/) (2018)
- [DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset](https://mlsanity.com/paper/1710.03957/) (2017)
- [COMET: Commonsense Transformers for Automatic Knowledge Graph Construction](https://mlsanity.com/paper/1906.05317/) (2019)
- [Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders](https://mlsanity.com/paper/1703.10960/) (2017)
- [Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models](https://mlsanity.com/paper/1507.04808/) (2015)
- [A Neural Conversational Model](https://mlsanity.com/paper/1506.05869/) (2015)
- [Build it Break it Fix it for Dialogue Safety: Robustness from Adversarial Human Attack](https://mlsanity.com/paper/1908.06083/) (2019)
- [The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems](https://mlsanity.com/paper/1506.08909/) (2015)
- [Recipes for Safety in Open-domain Chatbots](https://mlsanity.com/paper/2010.07079/) (2020)
- [A Survey of Evaluation Metrics Used for NLG Systems](https://mlsanity.com/paper/2008.12009/) (2020)
- [Queens are Powerful too: Mitigating Gender Bias in Dialogue Generation](https://mlsanity.com/paper/1911.03842/) (2019)
- [Like hiking? You probably enjoy nature: Persona-grounded Dialog with Commonsense Expansions](https://mlsanity.com/paper/2010.03205/) (2020)
- [Investigating Evaluation of Open-Domain Dialogue Systems With Human Generated Multiple References](https://mlsanity.com/paper/1907.10568/) (2019)
- [Do Neural Dialog Systems Use the Conversation History Effectively? An Empirical Study](https://mlsanity.com/paper/1906.01603/) (2019)
- ["I'd rather just go to bed": Understanding Indirect Answers](https://mlsanity.com/paper/2010.03450/) (2020)
- [Neural Metaphor Detection in Context](https://mlsanity.com/paper/1808.09653/) (2018)
- [Open-Domain Conversational Agents: Current Progress, Open Problems, and Future Directions](https://mlsanity.com/paper/2006.12442/) (2020)
- [Probing Neural Dialog Models for Conversational Understanding](https://mlsanity.com/paper/2006.08331/) (2020)
- [Dialogue Generation on Infrequent Sentence Functions via Structured Meta-Learning](https://mlsanity.com/paper/2010.01495/) (2020)
- [Improving Automated Evaluation of Open Domain Dialog via Diverse Reference Augmentation](https://mlsanity.com/paper/2106.02833/) (2021)
- [From Solving a Problem Boldly to Cutting the Gordian Knot: Idiomatic Text Generation](https://mlsanity.com/paper/2104.06541/) (2021)

## Built on by (most cited first, 19 of 19)

- [FLUTE: Figurative Language Understanding through Textual Explanations](https://mlsanity.com/paper/2205.12404/) (2022)
- [Testing the Ability of Language Models to Interpret Figurative Language](https://mlsanity.com/paper/2204.12632/) (2022)
- [Sonnet or Not, Bot? Poetry Evaluation for Large Models and Datasets](https://mlsanity.com/paper/2406.18906/) (2024)
- [A Hard Nut to Crack: Idiom Detection with Conversational Large Language Models](https://mlsanity.com/paper/2405.10579/) (2024)
- [Are Representations Built from the Ground Up? An Empirical Examination of Local Composition in Language Models](https://mlsanity.com/paper/2210.03575/) (2022)
- [Enhancing Idiomatic Representation in Multiple Languages via an Adaptive Contrastive Triplet Loss](https://mlsanity.com/paper/2406.15175/) (2024)
- [State-of-the-art in Open-domain Conversational AI: A Survey](https://mlsanity.com/paper/2205.00965/) (2022)
- [EUREKA: EUphemism Recognition Enhanced through Knn-based methods and Augmentation](https://mlsanity.com/paper/2210.12846/) (2022)
- [Vector Representations of Idioms in Conversational Systems](https://mlsanity.com/paper/2205.03666/) (2022)
- [Getting BART to Ride the Idiomatic Train: Learning to Represent Idiomatic Expressions](https://mlsanity.com/paper/2207.03679/) (2022)
- [AI-Enabled Conversational Journaling for Advancing Parkinson's Disease Symptom Tracking](https://mlsanity.com/paper/2503.03532/) (2025)
- [Does the "most sinfully decadent cake ever" taste good? Answering Yes/No Questions from Figurative Contexts](https://mlsanity.com/paper/2309.13748/) (2023)
- [Unified Representation for Non-compositional and Compositional Expressions](https://mlsanity.com/paper/2310.19127/) (2023)
- [I Came, I Saw, I Explained: Benchmarking Multimodal LLMs on Figurative Meaning in Memes](https://mlsanity.com/paper/2603.23229/) (2026)
- [A Major Obstacle for NLP Research: Let's Talk about Time Allocation!](https://mlsanity.com/paper/2211.16858/) (2022)
- [Generating Continuations in Multilingual Idiomatic Contexts](https://mlsanity.com/paper/2310.20195/) (2023)
- [IEKG: A Commonsense Knowledge Graph for Idiomatic Expressions](https://mlsanity.com/paper/2312.06053/) (2023)
- [Systematic word meta-sense extension](https://mlsanity.com/paper/2311.13029/) (2023)
- [Evaluating Large Language Models on Multiword Expressions in Multilingual and Code-Switched Contexts](https://mlsanity.com/paper/2504.20051/) (2025)

## Abstract (arXiv)

Humans often employ figurative language use in communication, including during interactions with dialog systems. Thus, it is important for real-world dialog systems to be able to handle popular figurative language constructs like metaphor and simile. In this work, we analyze the performance of existing dialog models in situations where the input dialog context exhibits use of figurative language. We observe large gaps in handling of figurative language when evaluating the models on two open domain dialog datasets. When faced with dialog contexts consisting of figurative language, some models show very large drops in performance compared to contexts without figurative language. We encourage future research in dialog modeling to separately analyze and report results on figurative language in order to better test model capabilities relevant to real-world use. Finally, we propose lightweight solutions to help existing models become more robust to figurative language by simply using an external resource to translate figurative language to literal (non-figurative) forms while preserving the meaning to the best extent possible.