# Cross-language Sentence Selection via Data Augmentation and Rationale Training

arXiv: https://arxiv.org/abs/2106.02293 · HTML: https://mlsanity.com/paper/2106.02293/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 53 references that are in the map.
- First posted on arXiv 4 Jun 2021.

## Builds on (most cited first, 22 of 22)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Attention is not Explanation](https://mlsanity.com/paper/1902.10186/) (2019)
- [Reading Wikipedia to Answer Open-Domain Questions](https://mlsanity.com/paper/1704.00051/) (2017)
- [Word Translation Without Parallel Data](https://mlsanity.com/paper/1710.04087/) (2017)
- [Learning Word Vectors for 157 Languages](https://mlsanity.com/paper/1802.06893/) (2018)
- [A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings](https://mlsanity.com/paper/1805.06297/) (2018)
- [Loss in Translation: Learning Bilingual Word Mapping with a Retrieval Criterion](https://mlsanity.com/paper/1804.07745/) (2018)
- [Improving zero-shot learning by mitigating the hubness problem](https://mlsanity.com/paper/1412.6568/) (2014)
- [Deriving Machine Attention from Human Rationales](https://mlsanity.com/paper/1808.09367/) (2018)
- [Quasar: Datasets for Question Answering by Search and Reading](https://mlsanity.com/paper/1707.03904/) (2017)
- [Unsupervised Pretraining for Sequence to Sequence Learning](https://mlsanity.com/paper/1611.02683/) (2016)
- [Dual Conditional Cross-Entropy Filtering of Noisy Parallel Corpora](https://mlsanity.com/paper/1809.00197/) (2018)
- [Query Focused Abstractive Summarization: Incorporating Query Relevance, Multi-Document Coverage, and Summary Length Constraints into seq2seq Models](https://mlsanity.com/paper/1801.07704/) (2018)
- [Guided Alignment Training for Topic-Aware Neural Machine Translation](https://mlsanity.com/paper/1607.01628/) (2016)
- [Unsupervised Cross-Lingual Information Retrieval using Monolingual Data Only](https://mlsanity.com/paper/1805.00879/) (2018)
- [Bi-Directional Neural Machine Translation with Synthetic Parallel Data](https://mlsanity.com/paper/1805.11213/) (2018)
- [Improving Low-Resource Cross-lingual Document Retrieval by Reranking with Deep Bilingual Representations](https://mlsanity.com/paper/1906.03492/) (2019)
- [Modeling Diverse Relevance Patterns in Ad-hoc Retrieval](https://mlsanity.com/paper/1805.05737/) (2018)
- [A Strong Baseline for Learning Cross-Lingual Word Embeddings from Sentence Alignments](https://mlsanity.com/paper/1608.05426/) (2016)

## Built on by (most cited first, 5 of 5)

- [Data Augmentation Approaches in Natural Language Processing: A Survey](https://mlsanity.com/paper/2110.01852/) (2021)
- [XSemPLR: Cross-Lingual Semantic Parsing in Multiple Natural Languages and Meaning Representations](https://mlsanity.com/paper/2306.04085/) (2023)
- [Cross-Lingual Training with Dense Retrieval for Document Retrieval](https://mlsanity.com/paper/2109.01628/) (2021)
- [UniPSDA: Unsupervised Pseudo Semantic Data Augmentation for Zero-Shot Cross-Lingual Natural Language Understanding](https://mlsanity.com/paper/2406.16372/) (2024)
- [Query-oriented Data Augmentation for Session Search](https://mlsanity.com/paper/2407.03720/) (2024)

## Abstract (arXiv)

This paper proposes an approach to cross-language sentence selection in a low-resource setting. It uses data augmentation and negative sampling techniques on noisy parallel sentence data to directly learn a cross-lingual embedding-based query relevance model. Results show that this approach performs as well as or better than multiple state-of-the-art machine translation + monolingual retrieval systems trained on the same parallel data. Moreover, when a rationale training secondary objective is applied to encourage the model to match word alignment hints from a phrase-based statistical machine translation model, consistent improvements are seen across three language pairs (English-Somali, English-Swahili and English-Tagalog) over a variety of state-of-the-art baselines.