# Automatic Task Requirements Writing Evaluation via Machine Reading Comprehension

arXiv: https://arxiv.org/abs/2107.07957 · HTML: https://mlsanity.com/paper/2107.07957/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 32 references that are in the map.
- First posted on arXiv 15 Jul 2021.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [Adversarial Examples for Evaluating Reading Comprehension Systems](https://mlsanity.com/paper/1707.07328/) (2017)
- [An Introduction to Convolutional Neural Networks](https://mlsanity.com/paper/1511.08458/) (2015)
- [NewsQA: A Machine Comprehension Dataset](https://mlsanity.com/paper/1611.09830/) (2016)
- [Retrospective Reader for Machine Reading Comprehension](https://mlsanity.com/paper/2001.09694/) (2020)
- [Gated-Attention Readers for Text Comprehension](https://mlsanity.com/paper/1606.01549/) (2016)
- [WikiReading: A Novel Large-scale Language Understanding Task over Wikipedia](https://mlsanity.com/paper/1608.03542/) (2016)
- [Stochastic Answer Networks for Machine Reading Comprehension](https://mlsanity.com/paper/1712.03556/) (2017)
- [ReasoNet: Learning to Stop Reading in Machine Comprehension](https://mlsanity.com/paper/1609.05284/) (2016)
- [Text Understanding with the Attention Sum Reader Network](https://mlsanity.com/paper/1603.01547/) (2016)
- [Learning Multi-level Dependencies for Robust Word Recognition](https://mlsanity.com/paper/1911.09789/) (2019)
- [An Empirical Analysis of Multiple-Turn Reasoning Strategies in Reading Comprehension Tasks](https://mlsanity.com/paper/1711.03230/) (2017)

## Abstract (arXiv)

Task requirements (TRs) writing is an important question type in Key English Test and Preliminary English Test. A TR writing question may include multiple requirements and a high-quality essay must respond to each requirement thoroughly and accurately. However, the limited teacher resources prevent students from getting detailed grading instantly. The majority of existing automatic essay scoring systems focus on giving a holistic score but rarely provide reasons to support it. In this paper, we proposed an end-to-end framework based on machine reading comprehension (MRC) to address this problem to some extent. The framework not only detects whether an essay responds to a requirement question, but clearly marks where the essay answers the question. Our framework consists of three modules: question normalization module, ELECTRA based MRC module and response locating module. We extensively explore state-of-the-art MRC methods. Our approach achieves 0.93 accuracy score and 0.85 F1 score on a real-world educational dataset. To encourage reproducible results, we make our code publicly available at urlhttps://github.com/aied2021TRMRC/AIED2021TRMRCcode.