Figurative Language in Recognizing Textual Entailment
Not on the map. Why?
Abstract (arXiv)
We introduce a collection of recognizing textual entailment (RTE) datasets focused on figurative language. We leverage five existing datasets annotated for a variety of figurative language -- simile, metaphor, and irony -- and frame them into over 12,500 RTE examples.We evaluate how well state-of-the-art models trained on popular RTE datasets capture different aspects of figurative language. Our results and analyses indicate that these models might not sufficiently capture figurative language, struggling to perform pragmatic inference and reasoning about world knowledge. Ultimately, our datasets provide a challenging testbed for evaluating RTE models.
Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 15 of its 50 references that are in the map. First posted on arXiv 2 Jun 2021.
Builds on
15 of 50 references.
References · 15
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
- 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
- 2015A large annotated corpus for learning natural language inferencecited by 1,837
- 2017Supervised Learning of Universal Sentence Representations from Natural Language Inference Datacited by 428
- 2018Stress Test Evaluation for Natural Language Inferencecited by 211
- 2018Breaking NLI Systems with Sentences that Require Simple Lexical Inferencescited by 134
- 2020Are Natural Language Inference Models IMPPRESsive? Learning IMPlicature and PRESuppositioncited by 76
- 2019Probing Natural Language Inference Models through Semantic Fragmentscited by 64
- 2021MERMAID: Metaphor Generation with Symbolism and Discriminative Decodingcited by 49
- 2019Probing What Different NLP Tasks Teach Machines about Function Word Comprehensioncited by 44
- 2018Evaluating Compositionality in Sentence Embeddingscited by 35
- 2020Do Neural Models Learn Systematicity of Monotonicity Inference in Natural Language?cited by 29
Built on by
Citing papers per year: 2021: 1; 2022: 7; 2023: 6; 2024: 4; 2025: 4; 2026: 4.
26 papers.
Sort:
By area:
- 2023Can Large Language Models Transform Computational Social Science?cited by 245
- 2022FLUTE: Figurative Language Understanding through Textual Explanationscited by 78
- 2022Testing the Ability of Language Models to Interpret Figurative Languagecited by 67
- 2023Natural Language Reasoning, A Surveycited by 61
- 2023MemeCap: A Dataset for Captioning and Interpreting Memescited by 46
- 2021It's not Rocket Science : Interpreting Figurative Language in Narrativescited by 39
- 2024STBLLM: Breaking the 1-Bit Barrier with Structured Binary LLMscited by 31
- 2022Can Pre-trained Language Models Interpret Similes as Smart as Human?cited by 13
- 2022Improving Large-scale Paraphrase Acquisition and Generationcited by 8
- 2022Just-DREAM-about-it: Figurative Language Understanding with DREAM-FLUTEcited by 8
- 2024A Novel Cartography-Based Curriculum Learning Method Applied on RoNLI: The First Romanian Natural Language Inference Corpuscited by 8
- 2025Metaphor and Large Language Models: When Surface Features Matter More than Deep Understandingcited by 8
14 more
- 2023Evaluating Paraphrastic Robustness in Textual Entailment Modelscited by 7
- 2024Meta4XNLI: A Crosslingual Parallel Corpus for Metaphor Detection and Interpretationcited by 5
- 2025Progressive Binarization with Semi-Structured Pruning for LLMscited by 5
- 2022Getting BART to Ride the Idiomatic Train: Learning to Represent Idiomatic Expressionscited by 4
- 2025BTC-LLM: Efficient Sub-1-Bit LLM Quantization via Learnable Transformation and Binary Codebookcited by 4
- 2022Detecting Euphemisms with Literal Descriptions and Visual Imagerycited by 3
- 2026Preferences for Idiomatic Language are Acquired Slowly -- and Forgotten Quickly: A Case Study on Swedishcited by 3
- 2023Does the "most sinfully decadent cake ever" taste good? Answering Yes/No Questions from Figurative Contextscited by 2
- 2025Large Vision-Language Models for Knowledge-Grounded Data Annotation of Memescited by 2
- 2023Generating Continuations in Multilingual Idiomatic Contextscited by 1
- 2024MetaphorShare: A Dynamic Collaborative Repository of Open Metaphor Datasetscited by 1
- 2026D2Quant: Accurate Low-bit Post-Training Weight Quantization for LLMscited by 1
- 2026SinFoS: A Parallel Dataset for Translating Sinhala Figures of Speechcited by 1
- 2026As Easy as Rocket Science: Assessing the Ability of Large Language Models to Interpret Negation in Figurative Languagecited by 0
- 2026As Easy as Rocket Science: Assessing the Ability of Large Language Models to Interpret Negation in Figurative Language17 Jun 2026
- 2026SinFoS: A Parallel Dataset for Translating Sinhala Figures of Speech9 Feb 2026
- 2026Preferences for Idiomatic Language are Acquired Slowly -- and Forgotten Quickly: A Case Study on Swedish3 Feb 2026
- 2026D2Quant: Accurate Low-bit Post-Training Weight Quantization for LLMs30 Jan 2026
- 2025Metaphor and Large Language Models: When Surface Features Matter More than Deep Understanding21 Jul 2025
- 2025BTC-LLM: Efficient Sub-1-Bit LLM Quantization via Learnable Transformation and Binary Codebook24 May 2025
- 2025Progressive Binarization with Semi-Structured Pruning for LLMs3 Feb 2025
- 2025Large Vision-Language Models for Knowledge-Grounded Data Annotation of Memes23 Jan 2025
- 2024MetaphorShare: A Dynamic Collaborative Repository of Open Metaphor Datasets27 Nov 2024
- 2024STBLLM: Breaking the 1-Bit Barrier with Structured Binary LLMs3 Aug 2024
- 2024A Novel Cartography-Based Curriculum Learning Method Applied on RoNLI: The First Romanian Natural Language Inference Corpus20 May 2024
- 2024Meta4XNLI: A Crosslingual Parallel Corpus for Metaphor Detection and Interpretation10 Apr 2024
14 more
- 2023Generating Continuations in Multilingual Idiomatic Contexts31 Oct 2023
- 2023Does the "most sinfully decadent cake ever" taste good? Answering Yes/No Questions from Figurative Contexts24 Sep 2023
- 2023Evaluating Paraphrastic Robustness in Textual Entailment Models29 Jun 2023
- 2023MemeCap: A Dataset for Captioning and Interpreting Memes23 May 2023
- 2023Can Large Language Models Transform Computational Social Science?12 Apr 2023
- 2023Natural Language Reasoning, A Survey26 Mar 2023
- 2022Detecting Euphemisms with Literal Descriptions and Visual Imagery8 Nov 2022
- 2022Just-DREAM-about-it: Figurative Language Understanding with DREAM-FLUTE28 Oct 2022
- 2022Improving Large-scale Paraphrase Acquisition and Generation6 Oct 2022
- 2022Getting BART to Ride the Idiomatic Train: Learning to Represent Idiomatic Expressions8 Jul 2022
- 2022FLUTE: Figurative Language Understanding through Textual Explanations24 May 2022
- 2022Testing the Ability of Language Models to Interpret Figurative Language26 Apr 2022
- 2022Can Pre-trained Language Models Interpret Similes as Smart as Human?16 Mar 2022
- 2021It's not Rocket Science : Interpreting Figurative Language in Narratives31 Aug 2021
- 2023Natural Language Reasoning, A Surveycited by 61
- 2023Natural Language Reasoning, A Survey26 Mar 2023
- 2023Can Large Language Models Transform Computational Social Science?cited by 245
- 2022FLUTE: Figurative Language Understanding through Textual Explanationscited by 78
- 2022Testing the Ability of Language Models to Interpret Figurative Languagecited by 67
- 2023MemeCap: A Dataset for Captioning and Interpreting Memescited by 46
- 2021It's not Rocket Science : Interpreting Figurative Language in Narrativescited by 39
- 2024STBLLM: Breaking the 1-Bit Barrier with Structured Binary LLMscited by 31
- 2022Can Pre-trained Language Models Interpret Similes as Smart as Human?cited by 13
- 2022Improving Large-scale Paraphrase Acquisition and Generationcited by 8
- 2022Just-DREAM-about-it: Figurative Language Understanding with DREAM-FLUTEcited by 8
- 2024A Novel Cartography-Based Curriculum Learning Method Applied on RoNLI: The First Romanian Natural Language Inference Corpuscited by 8
- 2025Metaphor and Large Language Models: When Surface Features Matter More than Deep Understandingcited by 8
- 2023Evaluating Paraphrastic Robustness in Textual Entailment Modelscited by 7
12 most cited of 24 All 24 (JSON)
- 2026As Easy as Rocket Science: Assessing the Ability of Large Language Models to Interpret Negation in Figurative Language17 Jun 2026
- 2026SinFoS: A Parallel Dataset for Translating Sinhala Figures of Speech9 Feb 2026
- 2026Preferences for Idiomatic Language are Acquired Slowly -- and Forgotten Quickly: A Case Study on Swedish3 Feb 2026
- 2026D2Quant: Accurate Low-bit Post-Training Weight Quantization for LLMs30 Jan 2026
- 2025Metaphor and Large Language Models: When Surface Features Matter More than Deep Understanding21 Jul 2025
- 2025BTC-LLM: Efficient Sub-1-Bit LLM Quantization via Learnable Transformation and Binary Codebook24 May 2025
- 2025Progressive Binarization with Semi-Structured Pruning for LLMs3 Feb 2025
- 2024MetaphorShare: A Dynamic Collaborative Repository of Open Metaphor Datasets27 Nov 2024
- 2024STBLLM: Breaking the 1-Bit Barrier with Structured Binary LLMs3 Aug 2024
- 2024A Novel Cartography-Based Curriculum Learning Method Applied on RoNLI: The First Romanian Natural Language Inference Corpus20 May 2024
- 2024Meta4XNLI: A Crosslingual Parallel Corpus for Metaphor Detection and Interpretation10 Apr 2024
- 2023Generating Continuations in Multilingual Idiomatic Contexts31 Oct 2023
12 newest of 24 All 24 (JSON)