mlsanityData to 14 Oct 2026

NLP-IIS@UT at SemEval-2021 Task 4: Machine Reading Comprehension using the Long Document Transformer

arXiv 2105.03775 ↗First posted 8 May 2021cs.CL cs.IR cs.ITcited by 1

Not on the map. Why?

Abstract (arXiv)

This paper presents a technical report of our submission to the 4th task of SemEval-2021, titled: Reading Comprehension of Abstract Meaning. In this task, we want to predict the correct answer based on a question given a context. Usually, contexts are very lengthy and require a large receptive field from the model. Thus, common contextualized language models like BERT miss fine representation and performance due to the limited capacity of the input tokens. To tackle this problem, we used the Longformer model to better process the sequences. Furthermore, we utilized the method proposed in the Longformer benchmark on Wikihop dataset which improved the accuracy on our task data from 23.01% and 22.95% achieved by the baselines for subtask 1 and 2, respectively, to 70.30% and 64.38%.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 11 of its 18 references that are in the map. First posted on arXiv 8 May 2021.

Builds on

11 of 18 references.

References · 11

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  3. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  4. 2020Longformer: The Long-Document Transformercited by 3,413
  5. 2018Deep contextualized word representationscited by 2,290
  6. 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,026
  7. 2014Distributed Representations of Sentences and Documentscited by 678
  8. 2017Constructing Datasets for Multi-hop Reading Comprehension Across Documentscited by 190
  9. 2020Conditionally Adaptive Multi-Task Learning: Improving Transfer Learning in NLP Using Fewer Parameters & Less Datacited by 50
  10. 2020A Survey on Machine Reading Comprehension Systemscited by 41
  11. 2018SDNet: Contextualized Attention-based Deep Network for Conversational Question Answeringcited by 30

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024Enhanced Labeling Technique for Reddit Text and Fine-Tuned Longformer Models for Classifying Depression Severity in English and Lugandacited by 1