# Evaluating Machine Common Sense via Cloze Testing

arXiv: https://arxiv.org/abs/2201.07902 · HTML: https://mlsanity.com/paper/2201.07902/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 9 references that are in the map.
- First posted on arXiv 19 Jan 2022.

## Builds on (most cited first, 2 of 2)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Calculating the similarity between words and sentences using a lexical database and corpus statistics](https://mlsanity.com/paper/1802.05667/) (2018)

## Built on by (most cited first, 1 of 1)

- [Just ClozE! A Novel Framework for Evaluating the Factual Consistency Faster in Abstractive Summarization](https://mlsanity.com/paper/2210.02804/) (2022)

## Abstract (arXiv)

Language models (LMs) show state of the art performance for common sense (CS) question answering, but whether this ability implies a human-level mastery of CS remains an open question. Understanding the limitations and strengths of LMs can help researchers improve these models, potentially by developing novel ways of integrating external CS knowledge. We devise a series of tests and measurements to systematically quantify their performance on different aspects of CS. We propose the use of cloze testing combined with word embeddings to measure the LM's robustness and confidence. Our results show than although language models tend to achieve human-like accuracy, their confidence is subpar. Future work can leverage this information to build more complex systems, such as an ensemble of symbolic and distributed knowledge.