Transformers on Multilingual Clause-Level Morphology
Not on the map. Why?
Abstract (arXiv)
This paper describes our winning systems in MRL: The 1st Shared Task on Multilingual Clause-level Morphology (EMNLP 2022 Workshop) designed by KUIS AI NLP team. We present our work for all three parts of the shared task: inflection, reinflection, and analysis. We mainly explore transformers with two approaches: (i) training models from scratch in combination with data augmentation, and (ii) transfer learning with prefix-tuning at multilingual morphological tasks. Data augmentation significantly improves performance for most languages in the inflection and reinflection tasks. On the other hand, Prefix-tuning on a pre-trained mGPT model helps us to adapt analysis tasks in low-data and multilingual settings. While transformer architectures with data augmentation achieved the most promising results for inflection and reinflection tasks, prefix-tuning on mGPT received the highest results for the analysis task. Our systems received 1st place in all three tasks in MRL 2022.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 16 of its 26 references that are in the map. First posted on arXiv 3 Nov 2022.
Builds on
16 of 26 references.
References · 16
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
- 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
- 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
- 2017Decoupled Weight Decay Regularizationcited by 1,722
- 2017Learning multiple visual domains with residual adapterscited by 553
- 2022mGPT: Few-Shot Learners Go Multilingualcited by 141
- 2020Applying the Transformer to Character-level Transductioncited by 43
- 2019Pushing the Limits of Low-Resource Morphological Inflectioncited by 29
- 2019Exact Hard Monotonic Attention for Character-Level Transductioncited by 22
4 more
- 2021Morphosyntactic Tagging with Pre-trained Language Models for Arabic and its Dialectscited by 14
- 2019A Simple Joint Model for Improved Contextual Neural Lemmatizationcited by 7
- 2019Two Birds with One Stone: Investigating Invertible Neural Networks for Inverse Problems in Morphologycited by 3
- 2018Character-Level Models versus Morphology in Semantic Role Labelingcited by 2
Built on by
Citing papers per year: 2023: 1.
1 paper.