mlsanityData to 7 Oct 2026

ePiC: Employing Proverbs in Context as a Benchmark for Abstract Language Understanding

arXiv 2109.06838 ↗First posted 14 Sep 2021cs.CLcited by 14

Not on the map. Why?

Abstract (arXiv)

While large language models have shown exciting progress on several NLP benchmarks, evaluating their ability for complex analogical reasoning remains under-explored. Here, we introduce a high-quality crowdsourced dataset of narratives for employing proverbs in context as a benchmark for abstract language understanding. The dataset provides fine-grained annotation of aligned spans between proverbs and narratives, and contains minimal lexical overlaps between narratives and proverbs, ensuring that models need to go beyond surface-level reasoning to succeed. We explore three tasks: (1) proverb recommendation and alignment prediction, (2) narrative generation for a given proverb and topic, and (3) identifying narratives with similar motifs. Our experiments show that neural language models struggle on these tasks compared to humans, and these tasks pose multiple learning challenges.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 28 of its 67 references that are in the map. First posted on arXiv 14 Sep 2021.

Builds on

28 of 67 references.

References · 28

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  6. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  7. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  8. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  9. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  10. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  11. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  12. 2017Decoupled Weight Decay Regularizationcited by 1,722
16 more
  1. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  2. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
  3. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 896
  4. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 770
  5. 2017Automated Hate Speech Detection and the Problem of Offensive Languagecited by 724
  6. 2016A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Storiescited by 413
  7. 2019Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasetscited by 173
  8. 2017Discovering Discrete Latent Topics with Neural Variational Inferencecited by 79
  9. 2019ChID: A Large-scale Chinese IDiom Dataset for Cloze Testcited by 65
  10. 2021MERMAID: Metaphor Generation with Symbolism and Discriminative Decodingcited by 49
  11. 2021It's not Rocket Science : Interpreting Figurative Language in Narrativescited by 39
  12. 2018Neural Metaphor Detection in Contextcited by 34
  13. 2019Topic-Aware Neural Keyphrase Generation for Social Media Languagecited by 33
  14. 2021Metaphor Generation with Conceptual Mappingscited by 32
  15. 2019Neural Simile Recognition with Cyclic Multitask Learning and Local Attentioncited by 18
  16. 2021Continuity of Topic, Interaction, and Query: Learning to Quote in Online Conversationscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 2; 2025: 3; 2026: 5.

14 papers.

Sort:

By area:

  1. 2023Are Multilingual LLMs Culturally-Diverse Reasoners? An Investigation into Multicultural Proverbs and Sayingscited by 32
  2. 2024AnaloBench: Benchmarking the Identification of Abstract and Long-context Analogiescited by 13
  3. 2023ANALOGICAL -- A Novel Benchmark for Long Text Analogy Evaluation in Large Language Modelscited by 12
  4. 2025Help Me Write a Story: Evaluating LLMs' Ability to Generate Writing Feedbackcited by 10
  5. 2023ARN: Analogical Reasoning on Narrativescited by 8
  6. 2025Pun Unintended: LLMs and the Illusion of Humor Understandingcited by 2
  7. 2023Object Recognition from Scientific Document based on Compartment Refinement Frameworkcited by 1
  8. 2024A Statistical Analysis of LLMs' Self-Evaluation Using Proverbscited by 1
  9. 2026SinFoS: A Parallel Dataset for Translating Sinhala Figures of Speechcited by 1
  10. 2025Proverbs or Pythian Oracles? Sentiments and Emotions in Greek Sayingscited by 0
  11. 2026Structural Abstraction as an Inductive Bias for Non-Stationary Language Model Trainingcited by 0
  12. 2026Constrained Semantic Decompression in LLMs through Persian Proverb-Conditioned Story Generationcited by 0
2 more
  1. 2026ADAGE: A Language-Agnostic Pipeline for Analogical Reasoning Evaluationcited by 0
  2. 2026Generating Constructive Feedback on Stories via Reinforcement Learningcited by 0