mlsanityData to 14 Oct 2026

CSSL: Contrastive Self-Supervised Learning for Dependency Parsing on Relatively Free Word Ordered and Morphologically Rich Low Resource Languages

arXiv 2410.06944 ↗First posted 9 Oct 2024cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Neural dependency parsing has achieved remarkable performance for low resource morphologically rich languages. It has also been well-studied that morphologically rich languages exhibit relatively free word order. This prompts a fundamental investigation: Is there a way to enhance dependency parsing performance, making the model robust to word order variations utilizing the relatively free word order nature of morphologically rich languages? In this work, we examine the robustness of graph-based parsing architectures on 7 relatively free word order languages. We focus on scrutinizing essential modifications such as data augmentation and the removal of position encoding required to adapt these architectures accordingly. To this end, we propose a contrastive self-supervised learning method to make the model robust to word order variations. Furthermore, our proposed modification demonstrates a substantial average gain of 3.03/2.95 points in 7 relatively free word order languages, as measured by the UAS/LAS Score metric when compared to the best performing baseline.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 46 references that are in the map. First posted on arXiv 9 Oct 2024.

Builds on

22 of 46 references.

References · 22

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  3. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  4. 2019Contrastive Multiview Codingcited by 1,301
  5. 2020Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillationcited by 593
  6. 2016Deep Biaffine Attention for Neural Dependency Parsingcited by 334
  7. 2020DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representationscited by 251
  8. 2021ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfercited by 231
  9. 2018An efficient framework for learning sentence representationscited by 198
  10. 2020CLEAR: Contrastive Learning for Sentence Representationcited by 186
  11. 2020CERT: Contrastive Self-supervised Learning for Language Understandingcited by 169
  12. 2018Semi-Supervised Sequence Modeling with Cross-View Trainingcited by 75
10 more
  1. 2019Data Augmentation via Dependency Tree Morphing for Low-Resource Languagescited by 45
  2. 2019Deep Contextualized Self-training for Low Resource Dependency Parsingcited by 24
  3. 2020Recursive Non-Autoregressive Graph-to-Graph Transformer for Dependency Parsing with Iterative Refinementcited by 20
  4. 2021Contrastive Domain Adaptation for Question Answering using Limited Text Corporacited by 20
  5. 2023A Cross-Linguistic Pressure for Uniform Information Density in Word Ordercited by 17
  6. 2019Graph-to-Graph Transformer for Transition-based Dependency Parsingcited by 13
  7. 2021A Little Pretraining Goes a Long Way: A Case Study on Dependency Parsing Task for Low-resource Morphologically Rich Languagescited by 9
  8. 2018An improved neural network model for joint POS tagging and dependency parsingcited by 8
  9. 2018What do character-level models learn about morphology? The case of dependency parsingcited by 8
  10. 2024A Morphology-Based Investigation of Positional Encodingscited by 4

Built on by

Not cited yet.