mlsanityData to 7 Oct 2026

75 Languages, 1 Model: Parsing Universal Dependencies Universally

arXiv 1904.02099 ↗First posted 3 Apr 2019cs.CL cs.LGcited by 90

Not on the map. Why?

Abstract (arXiv)

We present UDify, a multilingual multi-task model capable of accurately predicting universal part-of-speech, morphological features, lemmas, and dependency trees simultaneously for all 124 Universal Dependencies treebanks across 75 languages. By leveraging a multilingual BERT self-attention model pretrained on 104 languages, we found that fine-tuning it on all datasets concatenated together with simple softmax classifiers for each UD task can result in state-of-the-art UPOS, UFeats, Lemmas, UAS, and LAS scores, without requiring any recurrent or language-specific components. We evaluate UDify for multilingual learning, showing that low-resource languages benefit the most from cross-linguistic annotations. We also evaluate for zero-shot learning, with results suggesting that multilingual training provides strong UD predictions even for languages that neither UDify nor BERT have ever been trained on. Code for UDify is available at https://github.com/hyperparticle/udify.

Key facts. Cited by 90 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 32; 2022: 22; 2023: 17; 2024: 12; 2025: 1; 2026: 6.

90 papers.

Sort:

By area:

  1. 2023ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learningcited by 203
  2. 2023Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedbackcited by 189
  3. 2021Systematic Inequalities in Language Technology Performance across the World's Languagescited by 118
  4. 2021X-FACT: A New Benchmark Dataset for Multilingual Fact Checkingcited by 90
  5. 2021Trankit: A Light-Weight Transformer-based Toolkit for Multilingual Natural Language Processingcited by 68
  6. 2021One Question Answering Model for Many Languages with Cross-lingual Dense Passage Retrievalcited by 46
  7. 2022Expanding Pretrained Models to Thousands More Languages via Lexicon-based Adaptationcited by 44
  8. 2021RobeCzech: Czech RoBERTa, a monolingual contextualized language representation modelcited by 33
  9. 2022Analyzing the Mono- and Cross-Lingual Pretraining Dynamics of Multilingual Language Modelscited by 32
  10. 2021The Stem Cell Hypothesis: Dilemma behind Multi-Task Learning with Transformer Encoderscited by 31
  11. 2023Zero-shot cross-lingual transfer language selection using linguistic similaritycited by 28
  12. 2023NorBench -- A Benchmark for Norwegian Language Modelscited by 23
38 more
  1. 2021Attention Can Reflect Syntactic Structure (If You Let It)cited by 22
  2. 2021Specializing Multilingual Language Models: An Empirical Studycited by 21
  3. 2023Not all layers are equally as important: Every Layer Counts BERTcited by 21
  4. 2021Cross-lingual Visual Pre-training for Multimodal Machine Translationcited by 19
  5. 2021Subword Pooling Makes a Differencecited by 18
  6. 2022Protein Language Models and Structure Prediction: Connection and Progressioncited by 17
  7. 2024One Model is All You Need: ByT5-Sanskrit, a Unified Model for Sanskrit NLP Taskscited by 15
  8. 2022Pre-training Data Quality and Quantity for a Low-Resource Language: New Corpus and BERT Models for Maltesecited by 14
  9. 2021PhoNLP: A joint multi-task learning model for Vietnamese part-of-speech tagging, named entity recognition and dependency parsingcited by 13
  10. 2022Skill Extraction from Job Postings using Weak Supervisioncited by 13
  11. 2021Capitalization and Punctuation Restoration: a Surveycited by 12
  12. 2021Meta-Learning for Fast Cross-Lingual Adaptation in Dependency Parsingcited by 10
  13. 2023Extending the Pre-Training of BLOOM for Improved Support of Traditional Chinese: Models, Methods and Resultscited by 10
  14. 2023Taqyim: Evaluating Arabic NLP Tasks Using ChatGPT Modelscited by 10
  15. 2024mCSQA: Multilingual Commonsense Reasoning Dataset with Unified Creation Strategy by Language Models and Humanscited by 10
  16. 2021A Little Pretraining Goes a Long Way: A Case Study on Dependency Parsing Task for Low-resource Morphologically Rich Languagescited by 9
  17. 2021Evaluating the Morphosyntactic Well-formedness of Generated Textscited by 8
  18. 2021Genre as Weak Supervision for Cross-lingual Dependency Parsingcited by 8
  19. 2022Explanation Graph Generation via Pre-trained Language Models: An Empirical Study with Contrastive Learningcited by 8
  20. 2023Morphosyntactic probing of multilingual BERT modelscited by 8
  21. 2021Siamese BERT-based Model for Web Search Relevance Ranking Evaluated on a New Czech Datasetcited by 7
  22. 2021Model Compression for Domain Adaptation through Causal Effect Estimationcited by 6
  23. 2021Evaluating Transferability of BERT Models on Uralic Languagescited by 6
  24. 2021Cross-Lingual Open-Domain Question Answering with Answer Sentence Generationcited by 6
  25. 2022Yet Another Format of Universal Dependencies for Koreancited by 6
  26. 2021Zero-Shot Cross-Lingual Dependency Parsing through Contextual Embedding Transformationcited by 5
  27. 2021Joint Universal Syntactic and Semantic Parsingcited by 5
  28. 2022A Kind Introduction to Lexical and Grammatical Aspect, with a Survey of Computational Approachescited by 5
  29. 2023Transformers as Graph-to-Graph Modelscited by 5
  30. 2024Cross-Lingual Transfer from Related Languages: Treating Low-Resource Maltese as Multilingual Code-Switchingcited by 5
  31. 2021Coordinate Constructions in English Enhanced Universal Dependencies: Analysis and Computational Modelingcited by 4
  32. 2022Graph Neural Networks for Multiparallel Word Alignmentcited by 4
  33. 2022A Novel Multi-Task Learning Approach for Context-Sensitive Compound Type Identification in Sanskritcited by 4
  34. 2022Graph-Based Multilingual Label Propagation for Low-Resource Part-of-Speech Taggingcited by 4
  35. 2023Tokenization with Factorized Subword Encodingcited by 4
  36. 2024A Truly Joint Neural Architecture for Segmentation and Parsingcited by 4
  37. 2022Systematic Investigation of Strategies Tailored for Low-Resource Settings for Low-Resource Dependency Parsingcited by 3
  38. 2022AUTOLEX: An Automatic Framework for Linguistic Explorationcited by 3

50 most cited of 90 All 90 (JSON)