# Tapping BERT for Preposition Sense Disambiguation

arXiv: https://arxiv.org/abs/2111.13972 · HTML: https://mlsanity.com/paper/2111.13972/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 28 references that are in the map.
- First posted on arXiv 27 Nov 2021.

## Builds on (most cited first, 14 of 14)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Big Bird: Transformers for Longer Sequences](https://mlsanity.com/paper/2007.14062/) (2020)
- [How multilingual is Multilingual BERT?](https://mlsanity.com/paper/1906.01502/) (2019)
- [GlossBERT: BERT for Word Sense Disambiguation with Gloss Knowledge](https://mlsanity.com/paper/1908.07245/) (2019)
- [Does BERT Make Any Sense? Interpretable Word Sense Disambiguation with Contextualized Embeddings](https://mlsanity.com/paper/1909.10430/) (2019)
- [SemEval-2013 Task 4: Free Paraphrases of Noun Compounds](https://mlsanity.com/paper/1911.10421/) (2019)
- [Using BERT for Word Sense Disambiguation](https://mlsanity.com/paper/1909.08358/) (2019)
- [Cross-lingual Word Sense Disambiguation using mBERT Embeddings with Syntactic Dependencies](https://mlsanity.com/paper/2012.05300/) (2020)
- [The Preposition Project](https://mlsanity.com/paper/2104.08922/) (2021)
- [Semi Supervised Preposition-Sense Disambiguation using Multilingual Data](https://mlsanity.com/paper/1611.08813/) (2016)

## Abstract (arXiv)

Prepositions are frequently occurring polysemous words. Disambiguation of prepositions is crucial in tasks like semantic role labelling, question answering, text entailment, and noun compound paraphrasing. In this paper, we propose a novel methodology for preposition sense disambiguation (PSD), which does not use any linguistic tools. In a supervised setting, the machine learning model is presented with sentences wherein prepositions have been annotated with senses. These senses are IDs in what is called The Preposition Project (TPP). We use the hidden layer representations from pre-trained BERT and BERT variants. The latent representations are then classified into the correct sense ID using a Multi Layer Perceptron. The dataset used for this task is from SemEval-2007 Task-6. Our methodology gives an accuracy of 86.85% which is better than the state-of-the-art.