# CSSL: Contrastive Self-Supervised Learning for Dependency Parsing on Relatively Free Word Ordered and Morphologically Rich Low Resource Languages

arXiv: https://arxiv.org/abs/2410.06944 · HTML: https://mlsanity.com/paper/2410.06944/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 46 references that are in the map.
- First posted on arXiv 9 Oct 2024.

## Builds on (most cited first, 22 of 22)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation](https://mlsanity.com/paper/2004.09813/) (2020)
- [Deep Biaffine Attention for Neural Dependency Parsing](https://mlsanity.com/paper/1611.01734/) (2016)
- [DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations](https://mlsanity.com/paper/2006.03659/) (2020)
- [ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfer](https://mlsanity.com/paper/2105.11741/) (2021)
- [An efficient framework for learning sentence representations](https://mlsanity.com/paper/1803.02893/) (2018)
- [CLEAR: Contrastive Learning for Sentence Representation](https://mlsanity.com/paper/2012.15466/) (2020)
- [CERT: Contrastive Self-supervised Learning for Language Understanding](https://mlsanity.com/paper/2005.12766/) (2020)
- [Semi-Supervised Sequence Modeling with Cross-View Training](https://mlsanity.com/paper/1809.08370/) (2018)
- [Data Augmentation via Dependency Tree Morphing for Low-Resource Languages](https://mlsanity.com/paper/1903.09460/) (2019)
- [Deep Contextualized Self-training for Low Resource Dependency Parsing](https://mlsanity.com/paper/1911.04286/) (2019)
- [Recursive Non-Autoregressive Graph-to-Graph Transformer for Dependency Parsing with Iterative Refinement](https://mlsanity.com/paper/2003.13118/) (2020)
- [Contrastive Domain Adaptation for Question Answering using Limited Text Corpora](https://mlsanity.com/paper/2108.13854/) (2021)
- [A Cross-Linguistic Pressure for Uniform Information Density in Word Order](https://mlsanity.com/paper/2306.03734/) (2023)
- [Graph-to-Graph Transformer for Transition-based Dependency Parsing](https://mlsanity.com/paper/1911.03561/) (2019)
- [A Little Pretraining Goes a Long Way: A Case Study on Dependency Parsing Task for Low-resource Morphologically Rich Languages](https://mlsanity.com/paper/2102.06551/) (2021)
- [An improved neural network model for joint POS tagging and dependency parsing](https://mlsanity.com/paper/1807.03955/) (2018)
- [What do character-level models learn about morphology? The case of dependency parsing](https://mlsanity.com/paper/1808.09180/) (2018)
- [A Morphology-Based Investigation of Positional Encodings](https://mlsanity.com/paper/2404.04530/) (2024)

## Abstract (arXiv)

Neural dependency parsing has achieved remarkable performance for low resource morphologically rich languages. It has also been well-studied that morphologically rich languages exhibit relatively free word order. This prompts a fundamental investigation: Is there a way to enhance dependency parsing performance, making the model robust to word order variations utilizing the relatively free word order nature of morphologically rich languages? In this work, we examine the robustness of graph-based parsing architectures on 7 relatively free word order languages. We focus on scrutinizing essential modifications such as data augmentation and the removal of position encoding required to adapt these architectures accordingly. To this end, we propose a contrastive self-supervised learning method to make the model robust to word order variations. Furthermore, our proposed modification demonstrates a substantial average gain of 3.03/2.95 points in 7 relatively free word order languages, as measured by the UAS/LAS Score metric when compared to the best performing baseline.