mlsanityData to 14 Oct 2026

Cross-Lingual BERT Transformation for Zero-Shot Dependency Parsing

arXiv 1909.06775 ↗First posted 15 Sep 2019cs.CLcited by 46

Not on the map. Why?

Abstract (arXiv)

This paper investigates the problem of learning cross-lingual representations in a contextual space. We propose Cross-Lingual BERT Transformation (CLBT), a simple and efficient approach to generate cross-lingual contextualized word embeddings based on publicly available pre-trained BERT models (Devlin et al., 2018). In this approach, a linear transformation is learned from contextual word alignments to align the contextualized embeddings independently trained in different languages. We demonstrate the effectiveness of this approach on zero-shot cross-lingual transfer parsing. Experiments show that our embeddings substantially outperform the previous state-of-the-art that uses static embeddings. We further compare our approach with XLM (Lample and Conneau, 2019), a recently proposed cross-lingual language model trained with massive parallel data, and achieve highly competitive results.

Key facts. Cited by 46 papers in the map (data through 14 Oct 2026). First posted on arXiv 15 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 20; 2022: 12; 2023: 7; 2024: 4; 2025: 2; 2026: 1.

46 papers.

Sort:

By area:

  1. 2021Mitigating Language-Dependent Ethnic Bias in BERTcited by 87
  2. 2021NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentationcited by 73
  3. 2021It's All in the Heads: Using Attention Heads as a Baseline for Cross-Lingual Transfer in Commonsense Reasoningcited by 69
  4. 2021A Primer on Pretrained Multilingual Language Modelscited by 49
  5. 2021Consistency Regularization for Cross-Lingual Fine-Tuningcited by 41
  6. 2021Using Optimal Transport as Alignment Objective for fine-tuning Multilingual Contextualized Embeddingscited by 19
  7. 2023Exploring the Relationship between Alignment and Cross-lingual Transfer in Multilingual Transformerscited by 18
  8. 2021The NLP Cookbook: Modern Recipes for Transformer based Deep Learning Architecturescited by 16
  9. 2021Contributions of Transformer Attention Heads in Multi- and Cross-lingual Taskscited by 16
  10. 2022GL-CLeF: A Global-Local Contrastive Learning Framework for Cross-lingual Spoken Language Understandingcited by 14
  11. 2021Goldilocks: Just-Right Tuning of BERT for Technology-Assisted Reviewcited by 12
  12. 2021X-METRA-ADA: Cross-lingual Meta-Transfer Learning Adaptation to Natural Language Understanding and Question Answeringcited by 10
34 more
  1. 2022Analyzing Gender Representation in Multilingual Modelscited by 10
  2. 2021Unsupervised Multilingual Sentence Embeddings for Parallel Corpus Miningcited by 9
  3. 2023Fine-Tuning BERT with Character-Level Noise for Zero-Shot Transfer to Dialects and Closely-Related Languagescited by 9
  4. 2021El Volumen Louder Por Favor: Code-switching in Task-oriented Semantic Parsingcited by 8
  5. 2022Combining Static and Contextualised Multilingual Embeddingscited by 8
  6. 2022Transfer Language Selection for Zero-Shot Cross-Lingual Abusive Language Detectioncited by 8
  7. 2023Free Lunch: Robust Cross-Lingual Transfer via Model Checkpoint Averagingcited by 8
  8. 2022Por Qu'e N~ao Utiliser Alla Spraak? Mixed Training with Gradient Optimization in Few-Shot Cross-Lingual Transfercited by 7
  9. 2022IsoVec: Controlling the Relative Isomorphism of Word Embedding Spacescited by 7
  10. 2022Polyglot Prompt: Multilingual Multitask PrompTrainingcited by 6
  11. 2022ALIGN-MLM: Word Embedding Alignment is Crucial for Multilingual Pre-trainingcited by 6
  12. 2024Can Machine Translation Bridge Multilingual Pretraining and Cross-lingual Transfer Learning?cited by 6
  13. 2021Zero-Shot Cross-Lingual Dependency Parsing through Contextual Embedding Transformationcited by 5
  14. 2021Substructure Distribution Projection for Zero-Shot Cross-Lingual Dependency Parsingcited by 5
  15. 2022The Impact of Cross-Lingual Adjustment of Contextual Word Representations on Zero-Shot Transfercited by 5
  16. 2021On the Relation between Syntactic Divergence and Zero-Shot Performancecited by 4
  17. 2023Combating the Curse of Multilinguality in Cross-Lingual WSD by Aligning Sparse Contextualized Word Representationscited by 4
  18. 2023Improving Cross-Lingual Transfer through Subtree-Aware Word Reorderingcited by 4
  19. 2025ALGEN: Few-shot Inversion Attacks on Textual Embeddings using Alignment and Generationcited by 4
  20. 2021Multinational Address Parsing: A Zero-Shot Evaluationcited by 2
  21. 2022Cross-Lingual Phrase Retrievalcited by 2
  22. 2025AlignFreeze: Navigating the Impact of Realignment on the Layers of Multilingual Models Across Diverse Languagescited by 2
  23. 2021Improving Cross-Lingual Reading Comprehension with Self-Trainingcited by 1
  24. 2021Examining Cross-lingual Contextual Embeddings with Orthogonal Structural Probescited by 1
  25. 2021Transfer Learning for Multi-lingual Tasks -- a Surveycited by 1
  26. 2022A new approach to calculating BERTScore for automatic assessment of translation qualitycited by 1
  27. 2022Mismatching-Aware Unsupervised Translation Quality Estimation For Low-Resource Languagescited by 1
  28. 2023Measuring Catastrophic Forgetting in Cross-Lingual Transfer Paradigms: Exploring Tuning Strategiescited by 1
  29. 2024A Comparison of Language Modeling and Translation as Multilingual Pretraining Objectivescited by 1
  30. 2024Evaluating and explaining training strategies for zero-shot cross-lingual news sentiment analysiscited by 1
  31. 2026The Grammar of Transformers: A Systematic Review of Interpretability Research on Syntactic Knowledge in Language Modelscited by 1
  32. 2021Self-Attentive Constituency Parsing for UCCA-based Semantic Parsingcited by 0
  33. 2023I2KD-SLU: An Intra-Inter Knowledge Distillation Framework for Zero-Shot Cross-Lingual Spoken Language Understandingcited by 0
  34. 2024A survey of neural-network-based methods utilising comparable data for finding translation equivalentscited by 0