ZJUKLAB at SemEval-2021 Task 4: Negative Augmentation with Language Model for Reading Comprehension of Abstract Meaning
Not on the map. Why?
Abstract (arXiv)
This paper presents our systems for the three Subtasks of SemEval Task4: Reading Comprehension of Abstract Meaning (ReCAM). We explain the algorithms used to learn our models and the process of tuning the algorithms and selecting the best model. Inspired by the similarity of the ReCAM task and the language pre-training, we propose a simple yet effective technology, namely, negative augmentation with language model. Evaluation results demonstrate the effectiveness of our proposed approach. Our models achieve the 4th rank on both official test sets of Subtask 1 and Subtask 2 with an accuracy of 87.9% and an accuracy of 92.8%, respectively. We further conduct comprehensive model analysis and observe interesting error cases, which may promote future researches.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 25 of its 39 references that are in the map. First posted on arXiv 25 Feb 2021.
Builds on
25 of 39 references.
References · 25
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
- 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
- 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
- 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
- 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
- 2019Language Models as Knowledge Bases?cited by 1,922
- 2017Decoupled Weight Decay Regularizationcited by 1,722
- 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
13 more
- 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
- 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
- 2015Teaching Machines to Read and Comprehendcited by 1,039
- 2019When Does Label Smoothing Help?cited by 985
- 2018CoQA: A Conversational Question Answering Challengecited by 674
- 2020Contrastive Learning with Hard Negative Samplescited by 619
- 2019How to Fine-Tune BERT for Text Classification?cited by 409
- 2019Evaluating Commonsense in Pre-trained Language Modelscited by 95
- 2019Long-tail Relation Extraction via Knowledge Graph Embeddings and Graph Convolution Networkscited by 61
- 2016Machine Comprehension Using Match-LSTM and Answer Pointercited by 47
- 2019Relation Adversarial Network for Low Resource Knowledge Graph Completioncited by 24
- 2020Conceptualized Representation Learning for Chinese Biomedical Text Miningcited by 19
- 2020Bridging Text and Knowledge with Multi-Prototype Embedding for Few-Shot Relational Triple Extractioncited by 18
Built on by
Citing papers per year: 2021: 1; 2022: 1.
2 papers.