mlsanityData to 7 Oct 2026

A Little Pretraining Goes a Long Way: A Case Study on Dependency Parsing Task for Low-resource Morphologically Rich Languages

arXiv 2102.06551 ↗First posted 12 Feb 2021cs.CLcited by 9

Not on the map. Why?

Abstract (arXiv)

Neural dependency parsing has achieved remarkable performance for many domains and languages. The bottleneck of massive labeled data limits the effectiveness of these approaches for low resource languages. In this work, we focus on dependency parsing for morphological rich languages (MRLs) in a low-resource setting. Although morphological information is essential for the dependency parsing task, the morphological disambiguation and lack of powerful analyzers pose challenges to get this information for MRLs. To address these challenges, we propose simple auxiliary tasks for pretraining. We perform experiments on 10 MRLs in low-resource settings to measure the efficacy of our proposed pretraining method and observe an average absolute gain of 2 points (UAS) and 3.6 points (LAS). Code and data available at: https://github.com/jivnesh/LCM

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 20 of its 50 references that are in the map. First posted on arXiv 12 Feb 2021.

Builds on

20 of 50 references.

References · 20

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  3. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  4. 2018Deep contextualized word representationscited by 2,285
  5. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 789
  6. 2015Bidirectional LSTM-CRF Models for Sequence Taggingcited by 544
  7. 2016Deep Biaffine Attention for Neural Dependency Parsingcited by 334
  8. 2018Learning Word Vectors for 157 Languagescited by 279
  9. 2017Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasmcited by 117
  10. 201975 Languages, 1 Model: Parsing Universal Dependencies Universallycited by 90
  11. 2018Semi-Supervised Sequence Modeling with Cross-View Trainingcited by 75
  12. 2016Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representationscited by 55
8 more
  1. 2015Transition-Based Dependency Parsing with Stack Long Short-Term Memorycited by 45
  2. 2018Stack-Pointer Networks for Dependency Parsingcited by 36
  3. 2018NCRF++: An Open-source Neural Sequence Labeling Toolkitcited by 29
  4. 2019Deep Contextualized Self-training for Low Resource Dependency Parsingcited by 24
  5. 2019Deep Contextualized Word Embeddings in Transition-Based and Graph-Based Dependency Parsing -- A Tale of Two Parsers Revisitedcited by 13
  6. 2018An improved neural network model for joint POS tagging and dependency parsingcited by 8
  7. 2018What do character-level models learn about morphology? The case of dependency parsingcited by 8
  8. 2020Evaluating Neural Morphological Taggers for Sanskritcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 2; 2023: 2; 2024: 2; 2025: 1; 2026: 1.

9 papers.

  1. 2022TransLIST: A Transformer-Based Linguistically Informed Sanskrit Tokenizercited by 13
  2. 2021Evaluating Neural Word Embeddings for Sanskritcited by 8
  3. 2023SanskritShala: A Neural Sanskrit NLP Toolkit with Web-Based Interface for Pedagogical and Annotation Purposescited by 5
  4. 2024A Truly Joint Neural Architecture for Segmentation and Parsingcited by 4
  5. 2022Systematic Investigation of Strategies Tailored for Low-Resource Settings for Low-Resource Dependency Parsingcited by 3
  6. 2025Still Not There: Can LLMs Outperform Smaller Task-Specific Seq2Seq Models on the Poetry-to-Prose Conversion Task?cited by 2
  7. 2023Linguistically-Informed Neural Architectures for Lexical, Syntactic and Semantic Tasks in Sanskritcited by 1
  8. 2024CSSL: Contrastive Self-Supervised Learning for Dependency Parsing on Relatively Free Word Ordered and Morphologically Rich Low Resource Languagescited by 0
  9. 2026A Word-Level Digital Reader of the Prasthanatrayi with Sankara's Bhasya: Corpus, Method, and an Open, Offline Reading Aid for the Advaita Vedanta Canoncited by 0