ReCAM@IITK at SemEval-2021 Task 4: BERT and ALBERT based Ensemble for Abstract Word Prediction
Not on the map. Why?
Abstract (arXiv)
This paper describes our system for Task 4 of SemEval-2021: Reading Comprehension of Abstract Meaning (ReCAM). We participated in all subtasks where the main goal was to predict an abstract word missing from a statement. We fine-tuned the pre-trained masked language models namely BERT and ALBERT and used an Ensemble of these as our submitted system on Subtask 1 (ReCAM-Imperceptibility) and Subtask 2 (ReCAM-Nonspecificity). For Subtask 3 (ReCAM-Intersection), we submitted the ALBERT model as it gives the best results. We tried multiple approaches and found that Masked Language Modeling(MLM) based approach works the best.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 13 of its 21 references that are in the map. First posted on arXiv 4 Apr 2021.
Builds on
13 of 21 references.
References · 13
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
- 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,067
- 2015Teaching Machines to Read and Comprehendcited by 1,049
- 2015The Goldilocks Principle: Reading Children's Books with Explicit Memory Representationscited by 108
- 2019Hierarchical Transformers for Long Document Classificationcited by 104
- 2016Gated-Attention Readers for Text Comprehensioncited by 37
- 2018MCScript: A Novel Dataset for Assessing Machine Comprehension Using Script Knowledgecited by 30
1 more
Built on by
Not cited yet.