# Unsupervised Cross-Lingual Transfer of Structured Predictors without Source Data

arXiv: https://arxiv.org/abs/2110.03866 · HTML: https://mlsanity.com/paper/2110.03866/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 32 references that are in the map.
- First posted on arXiv 8 Oct 2021.

## Builds on (most cited first, 14 of 14)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [The State and Fate of Linguistic Diversity and Inclusion in the NLP World](https://mlsanity.com/paper/2004.09095/) (2020)
- [Stanza: A Python Natural Language Processing Toolkit for Many Human Languages](https://mlsanity.com/paper/2003.07082/) (2020)
- [Deep Biaffine Attention for Neural Dependency Parsing](https://mlsanity.com/paper/1611.01734/) (2016)
- [Offline bilingual word vectors, orthogonal transformations and the inverted softmax](https://mlsanity.com/paper/1702.03859/) (2017)
- [Universal Dependency Parsing from Scratch](https://mlsanity.com/paper/1901.10457/) (2019)
- [Torch-Struct: Deep Structured Prediction Library](https://mlsanity.com/paper/2002.00876/) (2020)
- [Single-/Multi-Source Cross-Lingual NER via Teacher-Student Learning on Unlabeled Data in Target Language](https://mlsanity.com/paper/2004.12440/) (2020)
- [Cross-Lingual Syntactic Transfer through Unsupervised Adaptation of Invertible Projections](https://mlsanity.com/paper/1906.02656/) (2019)
- [PPT: Parsimonious Parser Transfer for Unsupervised Cross-Lingual Adaptation](https://mlsanity.com/paper/2101.11216/) (2021)
- [Costs to Consider in Adopting NLP for Your Business](https://mlsanity.com/paper/2012.08958/) (2020)

## Abstract (arXiv)

Providing technologies to communities or domains where training data is scarce or protected e.g., for privacy reasons, is becoming increasingly important. To that end, we generalise methods for unsupervised transfer from multiple input models for structured prediction. We show that the means of aggregating over the input models is critical, and that multiplying marginal probabilities of substructures to obtain high-probability structures for distant supervision is substantially better than taking the union of such structures over the input models, as done in prior work. Testing on 18 languages, we demonstrate that the method works in a cross-lingual setting, considering both dependency parsing and part-of-speech structured prediction problems. Our analyses show that the proposed method produces less noisy labels for the distant supervision.