mlsanityData to 7 Oct 2026

Cross-Lingual Alignment of Contextual Word Embeddings, with Applications to Zero-shot Dependency Parsing

arXiv 1902.09492 ↗First posted 25 Feb 2019cs.CL cs.LGcited by 69

Not on the map. Why?

Abstract (arXiv)

We introduce a novel method for multilingual transfer that utilizes deep contextual embeddings, pretrained in an unsupervised fashion. While contextual embeddings have been shown to yield richer representations of meaning compared to their static counterparts, aligning them poses a challenge due to their dynamic nature. To this end, we construct context-independent variants of the original monolingual spaces and utilize their mapping to derive an alignment for the context-dependent spaces. This mapping readily supports processing of a target language, improving transfer by context-aware embeddings. Our experimental results demonstrate the effectiveness of this approach for zero-shot and few-shot learning of dependency parsing. Specifically, our method consistently outperforms the previous state-of-the-art on 6 tested languages, yielding an improvement of 6.8 LAS points on average.

Key facts. Cited by 69 papers in the map (data through 7 Oct 2026). First posted on arXiv 25 Feb 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 28; 2022: 10; 2023: 9; 2024: 16; 2025: 4; 2026: 2.

69 papers.

Sort:

By area:

  1. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 518
  2. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
  3. 2024Do Llamas Work in English? On the Latent Language of Multilingual Transformerscited by 275
  4. 2022Compute Trends Across Three Eras of Machine Learningcited by 161
  5. 2024A Survey on Multilingual Large Language Models: Corpora, Alignment, and Biascited by 82
  6. 2024The Semantic Hub Hypothesis: Language Models Share Semantic Representations Across Languages and Modalitiescited by 69
  7. 2021Multilingual Multimodal Pre-training for Zero-Shot Cross-Lingual Transfer of Vision-Language Modelscited by 41
  8. 2024Probing the Emergence of Cross-lingual Alignment during LLM Trainingcited by 38
  9. 2024Enhancing Multilingual Capabilities of Large Language Models through Self-Distillation from Resource-Rich Languagescited by 36
  10. 2024Language Imbalance Driven Rewarding for Multilingual Self-improvingcited by 27
  11. 2025Middle-Layer Representation Alignment for Cross-Lingual Transfer in Fine-Tuned LLMscited by 26
  12. 2024Bridging the Language Gaps in Large Language Models with Inference-Time Cross-Lingual Interventioncited by 24
38 more
  1. 2021Improving Zero-Shot Cross-Lingual Transfer Learning via Robust Trainingcited by 21
  2. 2021Zero-Shot Cross-lingual Semantic Parsingcited by 19
  3. 2021Using Optimal Transport as Alignment Objective for fine-tuning Multilingual Contextualized Embeddingscited by 19
  4. 2022MENLI: Robust Evaluation Metrics from Natural Language Inferencecited by 18
  5. 2021Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systemscited by 16
  6. 2022USCORE: An Effective Approach to Fully Unsupervised Evaluation Metrics for Machine Translationcited by 15
  7. 2024Breaking the Script Barrier in Multilingual Pre-Trained Language Models with Transliteration-Based Post-Training Alignmentcited by 14
  8. 2022Cluster & Tune: Boost Cold Start Performance in Text Classificationcited by 13
  9. 2021Combining Static Word Embeddings and Contextual Representations for Bilingual Lexicon Inductioncited by 12
  10. 2024Learn and Unlearn: Addressing Misinformation in Multilingual LLMscited by 11
  11. 2021Meta-Learning for Fast Cross-Lingual Adaptation in Dependency Parsingcited by 10
  12. 2021Unsupervised Multilingual Sentence Embeddings for Parallel Corpus Miningcited by 9
  13. 2021Exploring the Representation of Word Meanings in Context: A Case Study on Homonymy and Synonymycited by 9
  14. 2021Putting Words in BERT's Mouth: Navigating Contextualized Vector Spaces with Pseudowordscited by 9
  15. 2021El Volumen Louder Por Favor: Code-switching in Task-oriented Semantic Parsingcited by 8
  16. 2021Bootstrapping Multilingual AMR with Contextual Word Alignmentscited by 8
  17. 2022Computational historical linguistics and language diversity in South Asiacited by 8
  18. 2025Just Go Parallel: Improving the Multilingual Capabilities of Large Language Modelscited by 8
  19. 2021Cross-Lingual BERT Contextual Embedding Space Mapping with Isotropic and Isometric Conditionscited by 6
  20. 2022ALIGN-MLM: Word Embedding Alignment is Crucial for Multilingual Pre-trainingcited by 6
  21. 2023Are Structural Concepts Universal in Transformer Language Models? Towards Interpretable Cross-Lingual Generalizationcited by 6
  22. 2021Zero-Shot Cross-Lingual Dependency Parsing through Contextual Embedding Transformationcited by 5
  23. 2021Majority Voting with Bidirectional Pre-translation For Bitext Retrievalcited by 5
  24. 2021Joint Universal Syntactic and Semantic Parsingcited by 5
  25. 2021Soft Layer Selection with Meta-Learning for Zero-Shot Cross-Lingual Transfercited by 5
  26. 2021Substructure Distribution Projection for Zero-Shot Cross-Lingual Dependency Parsingcited by 5
  27. 2022The Impact of Cross-Lingual Adjustment of Contextual Word Representations on Zero-Shot Transfercited by 5
  28. 2024Lens: Rethinking Multilingual Enhancement for Large Language Modelscited by 5
  29. 2021Evaluation of contextual embeddings on less-resourced languagescited by 4
  30. 2021Cross-Lingual Text Classification of Transliterated Hindi and Malayalamcited by 4
  31. 2021On the Relation between Syntactic Divergence and Zero-Shot Performancecited by 4
  32. 2023Combating the Curse of Multilinguality in Cross-Lingual WSD by Aligning Sparse Contextualized Word Representationscited by 4
  33. 2023Improving Cross-Lingual Transfer through Subtree-Aware Word Reorderingcited by 4
  34. 2025ALGEN: Few-shot Inversion Attacks on Textual Embeddings using Alignment and Generationcited by 4
  35. 2021Towards Multi-Sense Cross-Lingual Alignment of Contextual Embeddingscited by 3
  36. 2021LAWDR: Language-Agnostic Weighted Document Representations from Pre-trained Modelscited by 3
  37. 2021Cross-lingual alignments of ELMo contextual embeddingscited by 3
  38. 2021Cross-lingual Transferring of Pre-trained Contextualized Language Modelscited by 3

50 most cited of 69 All 69 (JSON)