# Self-Augmented In-Context Learning for Unsupervised Word Translation

arXiv: https://arxiv.org/abs/2402.10024 · HTML: https://mlsanity.com/paper/2402.10024/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 36 references that are in the map.
- First posted on arXiv 15 Feb 2024.

## Builds on (most cited first, 24 of 24)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Quantifying the Carbon Emissions of Machine Learning](https://mlsanity.com/paper/1910.09700/) (2019)
- [Word Translation Without Parallel Data](https://mlsanity.com/paper/1710.04087/) (2017)
- [Probing Pretrained Language Models for Lexical Semantics](https://mlsanity.com/paper/2010.05731/) (2020)
- [Unsupervised Neural Machine Translation](https://mlsanity.com/paper/1710.11041/) (2017)
- [A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings](https://mlsanity.com/paper/1805.06297/) (2018)
- [Unsupervised Alignment of Embeddings with Wasserstein Procrustes](https://mlsanity.com/paper/1805.11222/) (2018)
- [Dictionary-based Phrase-level Prompting of Large Language Models for Machine Translation](https://mlsanity.com/paper/2302.07856/) (2023)
- [How to (Properly) Evaluate Cross-Lingual Word Embeddings: On Strong Baselines, Comparative Analyses, and Some Misconceptions](https://mlsanity.com/paper/1902.00508/) (2019)
- [Expanding Pretrained Models to Thousands More Languages via Lexicon-based Adaptation](https://mlsanity.com/paper/2203.09435/) (2022)
- [When Does Unsupervised Machine Translation Work?](https://mlsanity.com/paper/2004.05516/) (2020)
- [Bilingual Lexicon Induction with Semi-supervision in Non-Isometric Embedding Spaces](https://mlsanity.com/paper/1908.06625/) (2019)
- [Do We Really Need Fully Unsupervised Cross-Lingual Embeddings?](https://mlsanity.com/paper/1909.01638/) (2019)
- [Improving Word Translation via Two-Stage Contrastive Learning](https://mlsanity.com/paper/2203.08307/) (2022)
- [Improving the Lexical Ability of Pretrained Language Models for Unsupervised Neural Machine Translation](https://mlsanity.com/paper/2103.10531/) (2021)
- [Bilex Rx: Lexical Data Augmentation for Massively Multilingual Machine Translation](https://mlsanity.com/paper/2303.15265/) (2023)
- [Probing Cross-Lingual Lexical Knowledge from Multilingual Sentence Encoders](https://mlsanity.com/paper/2205.00267/) (2022)
- [Improving Bilingual Lexicon Induction with Cross-Encoder Reranking](https://mlsanity.com/paper/2210.16953/) (2022)

## Built on by (most cited first, 4 of 4)

- [CALRec: Contrastive Alignment of Generative LLMs for Sequential Recommendation](https://mlsanity.com/paper/2405.02429/) (2024)
- [Scaling Laws for Many-Shot In-Context Learning with Self-Generated Annotations](https://mlsanity.com/paper/2503.03062/) (2025)
- [Effective Self-Mining of In-Context Examples for Unsupervised Machine Translation with LLMs](https://mlsanity.com/paper/2410.11006/) (2024)
- [Self-Aug: Query and Entropy Adaptive Decoding for Large Vision-Language Models](https://mlsanity.com/paper/2510.13315/) (2025)

## Abstract (arXiv)

Recent work has shown that, while large language models (LLMs) demonstrate strong word translation or bilingual lexicon induction (BLI) capabilities in few-shot setups, they still cannot match the performance of 'traditional' mapping-based approaches in the unsupervised scenario where no seed translation pairs are available, especially for lower-resource languages. To address this challenge with LLMs, we propose self-augmented in-context learning (SAIL) for unsupervised BLI: starting from a zero-shot prompt, SAIL iteratively induces a set of high-confidence word translation pairs for in-context learning (ICL) from an LLM, which it then reapplies to the same LLM in the ICL fashion. Our method shows substantial gains over zero-shot prompting of LLMs on two established BLI benchmarks spanning a wide range of language pairs, also outperforming mapping-based baselines across the board. In addition to achieving state-of-the-art unsupervised BLI performance, we also conduct comprehensive analyses on SAIL and discuss its limitations.