# Domain Adaptation in Multilingual and Multi-Domain Monolingual Settings for Complex Word Identification

arXiv: https://arxiv.org/abs/2205.07283 · HTML: https://mlsanity.com/paper/2205.07283/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 48 references that are in the map.
- First posted on arXiv 15 May 2022.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels](https://mlsanity.com/paper/1805.07836/) (2018)
- [How multilingual is Multilingual BERT?](https://mlsanity.com/paper/1906.01502/) (2019)
- [A Brief Review of Domain Adaptation](https://mlsanity.com/paper/2010.03978/) (2020)
- [On Loss Functions for Deep Neural Networks in Classification](https://mlsanity.com/paper/1702.05659/) (2017)
- [Adversarial Deep Averaging Networks for Cross-Lingual Sentiment Classification](https://mlsanity.com/paper/1606.01614/) (2016)
- [A Report on the Complex Word Identification Shared Task 2018](https://mlsanity.com/paper/1804.09132/) (2018)
- [Adversarial Learning with Contextual Embeddings for Zero-resource Cross-lingual Classification and NER](https://mlsanity.com/paper/1909.00153/) (2019)
- [A Word-Complexity Lexicon and A Neural Readability Ranking Model for Lexical Simplification](https://mlsanity.com/paper/1810.05754/) (2018)
- [CompLex: A New Corpus for Lexical Complexity Prediction from Likert Scale Data](https://mlsanity.com/paper/2003.07008/) (2020)
- [SemEval-2021 Task 1: Lexical Complexity Prediction](https://mlsanity.com/paper/2106.00473/) (2021)
- [Predicting Lexical Complexity in English Texts: The Complex 2.0 Dataset](https://mlsanity.com/paper/2102.08773/) (2021)
- [Adversarial Training for Satire Detection: Controlling for Confounding Variables](https://mlsanity.com/paper/1902.11145/) (2019)
- [Strong Baselines for Complex Word Identification across Multiple Languages](https://mlsanity.com/paper/1904.05953/) (2019)
- [Domain Adversarial Fine-Tuning as an Effective Regularizer](https://mlsanity.com/paper/2009.13366/) (2020)
- [Cross-Lingual Transfer Learning for Complex Word Identification](https://mlsanity.com/paper/2010.01108/) (2020)
- [UPB at SemEval-2021 Task 1: Combining Deep Learning and Hand-Crafted Features for Lexical Complexity Prediction](https://mlsanity.com/paper/2104.06983/) (2021)
- [Improved Multi-Source Domain Adaptation by Preservation of Factors](https://mlsanity.com/paper/2010.07783/) (2020)

## Built on by (most cited first, 3 of 3)

- [State-of-the-art generalisation research in NLP: A taxonomy and review](https://mlsanity.com/paper/2210.03050/) (2022)
- [Towards Trustworthy and Aligned Machine Learning: A Data-centric Survey with Causality Perspectives](https://mlsanity.com/paper/2307.16851/) (2023)
- [Investigating Large Language Models for Complex Word Identification in Multilingual and Multidomain Setups](https://mlsanity.com/paper/2411.01706/) (2024)

## Abstract (arXiv)

Complex word identification (CWI) is a cornerstone process towards proper text simplification. CWI is highly dependent on context, whereas its difficulty is augmented by the scarcity of available datasets which vary greatly in terms of domains and languages. As such, it becomes increasingly more difficult to develop a robust model that generalizes across a wide array of input examples. In this paper, we propose a novel training technique for the CWI task based on domain adaptation to improve the target character and context representations. This technique addresses the problem of working with multiple domains, inasmuch as it creates a way of smoothing the differences between the explored datasets. Moreover, we also propose a similar auxiliary task, namely text simplification, that can be used to complement lexical complexity prediction. Our model obtains a boost of up to 2.42% in terms of Pearson Correlation Coefficients in contrast to vanilla training techniques, when considering the CompLex from the Lexical Complexity Prediction 2021 dataset. At the same time, we obtain an increase of 3% in Pearson scores, while considering a cross-lingual setup relying on the Complex Word Identification 2018 dataset. In addition, our model yields state-of-the-art results in terms of Mean Absolute Error.