# UPB at SemEval-2021 Task 1: Combining Deep Learning and Hand-Crafted Features for Lexical Complexity Prediction

arXiv: https://arxiv.org/abs/2104.06983 · HTML: https://mlsanity.com/paper/2104.06983/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 32 references that are in the map.
- First posted on arXiv 14 Apr 2021.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017)
- [Graph Convolutional Networks for Text Classification](https://mlsanity.com/paper/1809.05679/) (2018)
- [Adversarial Training for Aspect-Based Sentiment Analysis with BERT](https://mlsanity.com/paper/2001.11316/) (2020)
- [CompLex: A New Corpus for Lexical Complexity Prediction from Likert Scale Data](https://mlsanity.com/paper/2003.07008/) (2020)
- [Predicting Lexical Complexity in English Texts: The Complex 2.0 Dataset](https://mlsanity.com/paper/2102.08773/) (2021)
- [Strong Baselines for Complex Word Identification across Multiple Languages](https://mlsanity.com/paper/1904.05953/) (2019)
- [Complex Word Identification: Challenges in Data Annotation and System Performance](https://mlsanity.com/paper/1710.04989/) (2017)
- [Cross-Lingual Transfer Learning for Complex Word Identification](https://mlsanity.com/paper/2010.01108/) (2020)

## Built on by (most cited first, 5 of 5)

- [SemEval-2021 Task 1: Lexical Complexity Prediction](https://mlsanity.com/paper/2106.00473/) (2021)
- [Credible Plan-Driven RAG Method for Multi-Hop Question Answering](https://mlsanity.com/paper/2504.16787/) (2025)
- [Domain Adaptation in Multilingual and Multi-Domain Monolingual Settings for Complex Word Identification](https://mlsanity.com/paper/2205.07283/) (2022)
- [Lexical Complexity Prediction: An Overview](https://mlsanity.com/paper/2303.04851/) (2023)
- [Predicting Contextual Informativeness for Vocabulary Learning using Deep Learning](https://mlsanity.com/paper/2602.18326/) (2026)

## Abstract (arXiv)

Reading is a complex process which requires proper understanding of texts in order to create coherent mental representations. However, comprehension problems may arise due to hard-to-understand sections, which can prove troublesome for readers, while accounting for their specific language skills. As such, steps towards simplifying these sections can be performed, by accurately identifying and evaluating difficult structures. In this paper, we describe our approach for the SemEval-2021 Task 1: Lexical Complexity Prediction competition that consists of a mixture of advanced NLP techniques, namely Transformer-based language models, pre-trained word embeddings, Graph Convolutional Networks, Capsule Networks, as well as a series of hand-crafted textual complexity features. Our models are applicable on both subtasks and achieve good performance results, with a MAE below 0.07 and a Person correlation of .73 for single word identification, as well as a MAE below 0.08 and a Person correlation of .79 for multiple word targets. Our results are just 5.46% and 6.5% lower than the top scores obtained in the competition on the first and the second subtasks, respectively.