# Constructing Open Cloze Tests Using Generation and Discrimination Capabilities of Transformers

arXiv: https://arxiv.org/abs/2204.07237 · HTML: https://mlsanity.com/paper/2204.07237/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 30 references that are in the map.
- First posted on arXiv 14 Apr 2022.

## Builds on (most cited first, 5 of 5)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Large-scale Cloze Test Dataset Created by Teachers](https://mlsanity.com/paper/1711.03225/) (2017)
- [Manipulating the Difficulty of C-Tests](https://mlsanity.com/paper/1906.06905/) (2019)
- [Learning to Automatically Generate Fill-In-The-Blank Quizzes](https://mlsanity.com/paper/1806.04524/) (2018)

## Built on by (most cited first, 5 of 5)

- [On the application of Large Language Models for language teaching and assessment technology](https://mlsanity.com/paper/2307.08393/) (2023)
- [An overview of artificial intelligence in computer-assisted language learning](https://mlsanity.com/paper/2505.02032/) (2025)
- [AGReE: A system for generating Automated Grammar Reading Exercises](https://mlsanity.com/paper/2210.16302/) (2022)
- [Learning from Partially Annotated Data: Example-aware Creation of Gap-filling Exercises for Language Learning](https://mlsanity.com/paper/2306.01584/) (2023)
- [Controlling Cloze-test Question Item Difficulty with PLM-based Surrogate Models for IRT Assessment](https://mlsanity.com/paper/2403.01456/) (2024)

## Abstract (arXiv)

This paper presents the first multi-objective transformer model for constructing open cloze tests that exploits generation and discrimination capabilities to improve performance. Our model is further enhanced by tweaking its loss function and applying a post-processing re-ranking algorithm that improves overall test structure. Experiments using automatic and human evaluation show that our approach can achieve up to 82% accuracy according to experts, outperforming previous work and baselines. We also release a collection of high-quality open cloze tests along with sample system output and human annotations that can serve as a future benchmark.