mlsanityData to 14 Oct 2026

Cross-lingual Retrieval for Iterative Self-Supervised Training

arXiv 2006.09526 ↗First posted 16 Jun 2020cs.CL cs.LGcited by 54

Not on the map. Why?

Abstract (arXiv)

Recent studies have demonstrated the cross-lingual alignment ability of multilingual pretrained language models. In this work, we found that the cross-lingual alignment can be further improved by training seq2seq models on sentence pairs mined using their own encoder outputs. We utilized these findings to develop a new approach -- cross-lingual retrieval for iterative self-supervised training (CRISS), where mining and training processes are applied iteratively, improving cross-lingual alignment and translation ability at the same time. Using this method, we achieved state-of-the-art unsupervised machine translation results on 9 language directions with an average improvement of 2.4 BLEU, and on the Tatoeba sentence retrieval task in the XTREME benchmark on 16 languages with an average improvement of 21.5% in absolute accuracy. Furthermore, CRISS also brings an additional 1.8 BLEU improvement on average compared to mBART, when finetuned on supervised machine translation downstream tasks.

Key facts. Cited by 54 papers in the map (data through 14 Oct 2026). First posted on arXiv 16 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 23; 2022: 15; 2023: 7; 2024: 6; 2025: 2; 2026: 1.

54 papers.

Sort:

By area:

  1. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
  2. 2021Contrastive Learning for Many-to-many Multilingual Neural Machine Translationcited by 108
  3. 2021Facebook AI WMT21 News Translation Task Submissioncited by 57
  4. 2021End-to-end Speech Translation via Cross-modal Progressive Trainingcited by 47
  5. 2021One Question Answering Model for Many Languages with Cross-lingual Dense Passage Retrievalcited by 46
  6. 2023Democratizing LLMs for Low-Resource Languages by Leveraging their English Dominant Abilities with Linguistically-Diverse Promptscited by 45
  7. 2021CrossSum: Beyond English-Centric Cross-Lingual Summarization for 1,500+ Language Pairscited by 39
  8. 2022Toxicity Detection with Generative Prompt-based Inferencecited by 35
  9. 2023CiCo: Domain-Aware Sign Language Retrieval via Cross-Lingual Contrastive Learningcited by 33
  10. 2022English Contrastive Learning Can Learn Universal Cross-lingual Sentence Embeddingscited by 26
  11. 2022ByGPT5: End-to-End Style-conditioned Poetry Generation with Token-free Language Modelscited by 25
  12. 2021Zero-shot Cross-lingual Transfer of Neural Machine Translation with Multilingual Pretrained Encoderscited by 24
38 more
  1. 2021AfroMT: Pretraining Strategies and Reproducible Benchmarks for Translation of 8 African Languagescited by 24
  2. 2021Paraphrastic Representations at Scalecited by 21
  3. 2022CLASP: Few-Shot Cross-Lingual Data Augmentation for Semantic Parsingcited by 21
  4. 2021Bilingual Lexicon Induction via Unsupervised Bitext Construction and Word Alignmentcited by 19
  5. 2021Adapting High-resource NMT Models to Translate Low-resource Related Languages without Parallel Datacited by 17
  6. 2021On the Copying Behaviors of Pre-Training for Neural Machine Translationcited by 15
  7. 2021RETRONLU: Retrieval Augmented Task-Oriented Semantic Parsingcited by 15
  8. 2022USCORE: An Effective Approach to Fully Unsupervised Evaluation Metrics for Machine Translationcited by 15
  9. 2021On the Complementarity between Pre-Training and Back-Translation for Neural Machine Translationcited by 14
  10. 2023CLIPTrans: Transferring Visual Knowledge with Pre-trained Models for Multimodal Machine Translationcited by 14
  11. 2021A Dataset and Baselines for Multilingual Reply Suggestioncited by 9
  12. 2022Understanding and Mitigating the Uncertainty in Zero-Shot Translationcited by 9
  13. 2021A Massively Multilingual Analysis of Cross-linguality in Shared Embedding Spacecited by 8
  14. 2021Towards Making the Most of Multilingual Pretraining for Zero-Shot Neural Machine Translationcited by 8
  15. 2022Improving Speech-to-Speech Translation Through Unlabeled Textcited by 8
  16. 2025From Unaligned to Aligned: Scaling Multilingual LLMs with Multi-Way Parallel Corporacited by 7
  17. 2021Exploiting Curriculum Learning in Unsupervised Neural Machine Translationcited by 6
  18. 2021Majority Voting with Bidirectional Pre-translation For Bitext Retrievalcited by 5
  19. 2021"Wikily" Supervised Neural Translation Tailored to Cross-Lingual Taskscited by 5
  20. 2021Substructure Distribution Projection for Zero-Shot Cross-Lingual Dependency Parsingcited by 5
  21. 2022Bridging the Data Gap between Training and Inference for Unsupervised Neural Machine Translationcited by 5
  22. 2022Face-to-Face Contrastive Learning for Social Intelligence Question-Answeringcited by 5
  23. 2025Neuron-Guided Interpretation of Code LLMs: Where, Why, and How?cited by 5
  24. 2021X2Parser: Cross-Lingual and Cross-Domain Framework for Task-Oriented Compositional Semantic Parsingcited by 4
  25. 2024News Without Borders: Domain Adaptation of Multilingual Sentence Embeddings for Cross-lingual News Recommendationcited by 4
  26. 2026Mending the Holes: Mitigating Reward Hacking in Reinforcement Learning for Multilingual Translationcited by 4
  27. 2021Low-Resource Machine Translation Training Curriculum Fit for Low-Resource Languagescited by 3
  28. 2021BitextEdit: Automatic Bitext Editing for Improved Low-Resource Machine Translationcited by 3
  29. 2022OneAligner: Zero-shot Cross-lingual Transfer with One Rich-Resource Language Pair for Low-Resource Sentence Retrievalcited by 3
  30. 2022Refining Low-Resource Unsupervised Translation by Language Disentanglement of Multilingual Modelcited by 2
  31. 2023A Review on Objective-Driven Artificial Intelligencecited by 2
  32. 2024POMP: Probability-driven Meta-graph Prompter for LLMs in Low-resource Unsupervised Neural Machine Translationcited by 2
  33. 2022Few-shot Mining of Naturally Occurring Inputs and Outputscited by 1
  34. 2024Improving Cross-lingual Representation for Semantic Retrieval with Code-switchingcited by 1
  35. 2024Mitigating Semantic Leakage in Cross-lingual Embeddings via Orthogonality Constraintcited by 1
  36. 2022Finetuning a Kalaallisut-English machine translation system using web-crawled datacited by 0
  37. 2022LANS: Large-scale Arabic News Summarization Corpuscited by 0
  38. 2023PEACH: Pre-Training Sequence-to-Sequence Multilingual Models for Translation with Semi-Supervised Pseudo-Parallel Document Generationcited by 0

50 most cited of 54 All 54 (JSON)