# Does the "most sinfully decadent cake ever" taste good? Answering Yes/No Questions from Figurative Contexts

arXiv: https://arxiv.org/abs/2309.13748 · HTML: https://mlsanity.com/paper/2309.13748/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 36 references that are in the map.
- First posted on arXiv 24 Sep 2023.

## Builds on (most cited first, 13 of 13)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions](https://mlsanity.com/paper/1905.10044/) (2019)
- [Testing the Ability of Language Models to Interpret Figurative Language](https://mlsanity.com/paper/2204.12632/) (2022)
- [RiddleSense: Reasoning about Riddle Questions Featuring Linguistic Creativity and Commonsense Knowledge](https://mlsanity.com/paper/2101.00376/) (2021)
- [Can Transformer be Too Compositional? Analysing Idiom Processing in Neural Machine Translation](https://mlsanity.com/paper/2205.15301/) (2022)
- [Figurative Language in Recognizing Textual Entailment](https://mlsanity.com/paper/2106.01195/) (2021)
- [Investigating Robustness of Dialog Models to Popular Figurative Language Constructs](https://mlsanity.com/paper/2110.00687/) (2021)
- [Asking Too Much? The Rhetorical Role of Questions in Political Discourse](https://mlsanity.com/paper/1708.02254/) (2017)
- [From Solving a Problem Boldly to Cutting the Gordian Knot: Idiomatic Text Generation](https://mlsanity.com/paper/2104.06541/) (2021)
- [Harvesting Creative Templates for Generating Stylistically Varied Restaurant Reviews](https://mlsanity.com/paper/1709.05308/) (2017)

## Built on by (most cited first, 2 of 2)

- [Metaphor and Large Language Models: When Surface Features Matter More than Deep Understanding](https://mlsanity.com/paper/2507.15357/) (2025)
- [Meta4XNLI: A Crosslingual Parallel Corpus for Metaphor Detection and Interpretation](https://mlsanity.com/paper/2404.07053/) (2024)

## Abstract (arXiv)

Figurative language is commonplace in natural language, and while making communication memorable and creative, can be difficult to understand. In this work, we investigate the robustness of Question Answering (QA) models on figurative text. Yes/no questions, in particular, are a useful probe of figurative language understanding capabilities of large language models. We propose FigurativeQA, a set of 1000 yes/no questions with figurative and non-figurative contexts, extracted from the domains of restaurant and product reviews. We show that state-of-the-art BERT-based QA models exhibit an average performance drop of up to 15% points when answering questions from figurative contexts, as compared to non-figurative ones. While models like GPT-3 and ChatGPT are better at handling figurative texts, we show that further performance gains can be achieved by automatically simplifying the figurative contexts into their non-figurative (literal) counterparts. We find that the best overall model is ChatGPT with chain-of-thought prompting to generate non-figurative contexts. Our work provides a promising direction for building more robust QA models with figurative language understanding capabilities.