# Multilingual Extraction and Categorization of Lexical Collocations with Graph-aware Transformers

arXiv: https://arxiv.org/abs/2205.11456 · HTML: https://mlsanity.com/paper/2205.11456/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 76 references that are in the map.
- First posted on arXiv 23 May 2022.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [CamemBERT: a Tasty French Language Model](https://mlsanity.com/paper/1911.03894/) (2019)
- [Do Transformers Really Perform Bad for Graph Representation?](https://mlsanity.com/paper/2106.05234/) (2021)
- [BERT for Joint Intent Classification and Slot Filling](https://mlsanity.com/paper/1902.10909/) (2019)
- [Exploiting Similarities among Languages for Machine Translation](https://mlsanity.com/paper/1309.4168/) (2013)
- [PMI-Masking: Principled masking of correlated spans](https://mlsanity.com/paper/2010.01825/) (2020)
- [Still a Pain in the Neck: Evaluating Text Representations on Lexical Composition](https://mlsanity.com/paper/1902.10618/) (2019)
- [Multi-lingual Intent Detection and Slot Filling in a Joint BERT-based Model](https://mlsanity.com/paper/1907.02884/) (2019)
- [Recursive Non-Autoregressive Graph-to-Graph Transformer for Dependency Parsing with Iterative Refinement](https://mlsanity.com/paper/2003.13118/) (2020)
- [Robust Neural Machine Translation with Joint Textual and Phonetic Embedding](https://mlsanity.com/paper/1810.06729/) (2018)
- [Graph-to-Graph Transformer for Transition-based Dependency Parsing](https://mlsanity.com/paper/1911.03561/) (2019)
- [Syntax-Aware Graph-to-Graph Transformer for Semantic Role Labelling](https://mlsanity.com/paper/2104.07704/) (2021)
- [Boosting Span-based Joint Entity and Relation Extraction via Squence Tagging Mechanism](https://mlsanity.com/paper/2105.10080/) (2021)

## Built on by (most cited first, 2 of 2)

- [Transformers as Graph-to-Graph Models](https://mlsanity.com/paper/2310.17936/) (2023)
- [3D-EX : A Unified Dataset of Definitions and Dictionary Examples](https://mlsanity.com/paper/2308.03043/) (2023)

## Abstract (arXiv)

Recognizing and categorizing lexical collocations in context is useful for language learning, dictionary compilation and downstream NLP. However, it is a challenging task due to the varying degrees of frozenness lexical collocations exhibit. In this paper, we put forward a sequence tagging BERT-based model enhanced with a graph-aware transformer architecture, which we evaluate on the task of collocation recognition in context. Our results suggest that explicitly encoding syntactic dependencies in the model architecture is helpful, and provide insights on differences in collocation typification in English, Spanish and French.