mlsanityData to 7 Oct 2026

SemEval-2021 Task 4: Reading Comprehension of Abstract Meaning

arXiv 2105.14879 ↗First posted 31 May 2021cs.CLcited by 3

Not on the map. Why?

Abstract (arXiv)

This paper introduces the SemEval-2021 shared task 4: Reading Comprehension of Abstract Meaning (ReCAM). This shared task is designed to help evaluate the ability of machines in representing and understanding abstract concepts. Given a passage and the corresponding question, a participating system is expected to choose the correct answer from five candidates of abstract concepts in a cloze-style machine reading comprehension setup. Based on two typical definitions of abstractness, i.e., the imperceptibility and nonspecificity, our task provides three subtasks to evaluate the participating models. Specifically, Subtask 1 aims to evaluate how well a system can model concepts that cannot be directly perceived in the physical world. Subtask 2 focuses on models' ability in comprehending nonspecific concepts located high in a hypernym hierarchy given the context of a passage. Subtask 3 aims to provide some insights into models' generalizability over the two types of abstractness. During the SemEval-2021 official evaluation period, we received 23 submissions to Subtask 1 and 28 to Subtask 2. The participating teams additionally made 29 submissions to Subtask 3. The leaderboard and competition website can be found at https://competitions.codalab.org/competitions/26153. The data and baseline code are available at https://github.com/boyuanzheng010/SemEval2021-Reading-Comprehension-of-Abstract-Meaning.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 27 of its 45 references that are in the map. First posted on arXiv 31 May 2021.

Builds on

27 of 45 references.

References · 27

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  6. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  7. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  8. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  9. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  10. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  11. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  12. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
15 more
  1. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,194
  2. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,065
  3. 2015Teaching Machines to Read and Comprehendcited by 1,039
  4. 2020Stanza: A Python Natural Language Processing Toolkit for Many Human Languagescited by 719
  5. 2017The NarrativeQA Reading Comprehension Challengecited by 590
  6. 2016NewsQA: A Machine Comprehension Datasetcited by 330
  7. 2020Intermediate-Task Transfer Learning with Pretrained Models for Natural Language Understanding: When and Why Does It Work?cited by 127
  8. 2017DuReader: a Chinese Machine Reading Comprehension Dataset from Real-world Applicationscited by 110
  9. 2015The Goldilocks Principle: Reading Children's Books with Explicit Memory Representationscited by 108
  10. 2016Gated-Attention Readers for Text Comprehensioncited by 37
  11. 2016Who did What: A Large-Scale Person-Centered Cloze Datasetcited by 24
  12. 2021ZJUKLAB at SemEval-2021 Task 4: Negative Augmentation with Language Model for Reading Comprehension of Abstract Meaningcited by 2
  13. 2021LRG at SemEval-2021 Task 4: Improving Reading Comprehension with Abstract Words using Augmentation, Linguistic Features and Votingcited by 1
  14. 2021IIE-NLP-Eyas at SemEval-2021 Task 4: Enhancing PLM for ReCAM with Special Tokens, Re-Ranking, Siamese Encoders and Back Translationcited by 1
  15. 2021XRJL-HKUST at SemEval-2021 Task 4: WordNet-Enhanced Dual Multi-head Co-Attention for Reading Comprehension of Abstract Meaningcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 0; 2024: 0; 2025: 0; 2026: 1.

3 papers.

  1. 2022VLSP 2021 - ViMRC Challenge: Vietnamese Machine Reading Comprehensioncited by 7
  2. 2022Clozer: Adaptable Data Augmentation for Cloze-style Reading Comprehensioncited by 2
  3. 2026LLMs Struggle with Abstract Meaning Comprehension More Than Expectedcited by 0