Cross-lingual Contextualized Phrase Retrieval
Not on the map. Why?
Abstract (arXiv)
Phrase-level dense retrieval has shown many appealing characteristics in downstream NLP tasks by leveraging the fine-grained information that phrases offer. In our work, we propose a new task formulation of dense retrieval, cross-lingual contextualized phrase retrieval, which aims to augment cross-lingual applications by addressing polysemy using context information. However, the lack of specific training data and models are the primary challenges to achieve our goal. As a result, we extract pairs of cross-lingual phrases using word alignment information automatically induced from parallel sentences. Subsequently, we train our Cross-lingual Contextualized Phrase Retriever (CCPR) using contrastive learning, which encourages the hidden representations of phrases with similar contexts and semantics to align closely. Comprehensive experiments on both the cross-lingual phrase retrieval task and a downstream task, i.e, machine translation, demonstrate the effectiveness of CCPR. On the phrase retrieval task, CCPR surpasses baselines by a significant margin, achieving a top-1 accuracy that is at least 13 points higher. When utilizing CCPR to augment the large-language-model-based translator, it achieves average gains of 0.7 and 1.5 in BERTScore for translations from X=>En and vice versa, respectively, on WMT16 dataset. Our code and data are available at urlhttps://github.com/ghrua/ccprrelease.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 44 of its 66 references that are in the map. First posted on arXiv 25 Mar 2024.
Builds on
44 of 66 references.
References · 44
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
- 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
- 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
- 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
- 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
- 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
- 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
- 2021Improving language models by retrieving from trillions of tokenscited by 1,278
32 more
- 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
- 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,028
- 2020COMET: A Neural Framework for MT Evaluationcited by 1,024
- 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 853
- 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 632
- 2020Language-agnostic BERT Sentence Embeddingcited by 621
- 2020Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillationcited by 591
- 2021Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Modelscited by 462
- 2020Nearest Neighbor Machine Translationcited by 209
- 2021Word Alignment by Fine-tuning Embeddings on Parallel Corporacited by 186
- 2019Facebook FAIR's WMT19 News Translation Task Submissioncited by 179
- 2020SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddingscited by 155
- 2023Platypus: Quick, Cheap, and Powerful Refinement of LLMscited by 150
- 2019Learning Dense Representations for Entity Retrievalcited by 122
- 2020Learning Dense Representations of Phrases at Scalecited by 91
- 2019A Discrete Hard EM Approach for Weakly Supervised Question Answeringcited by 78
- 2018Guiding Neural Machine Translation with Retrieved Translation Piecescited by 63
- 2018On the Limitations of Unsupervised Bilingual Dictionary Inductioncited by 61
- 2021Neural Machine Translation with Monolingual Translation Memorycited by 61
- 2021Adaptive Nearest Neighbor Machine Translationcited by 58
- 2018Learning Cross-Lingual Sentence Representations via a Multi-task Dual-Encoder Modelcited by 46
- 2021Fast Nearest Neighbor Machine Translationcited by 37
- 2019Non-Parametric Adaptation for Neural Machine Translationcited by 34
- 2020Empirical Analysis of Unlabeled Entity Problem in Named Entity Recognitioncited by 34
- 2021Phrase Retrieval Learns Passage Retrieval, Toocited by 33
- 2017What does Attention in Neural Machine Translation Pay Attention to?cited by 30
- 2018Phrase-Indexed Question Answering: A New Challenge for Scalable Document Comprehensioncited by 23
- 2016Bridging Neural Machine Translation and Bilingual Dictionariescited by 18
- 2023WSPAlign: Word Alignment Pre-training via Large-Scale Weakly Supervised Span Predictioncited by 10
- 2023Multilingual Sentence Transformer as A Multilingual Word Alignercited by 8
- 2022Retrofitting Multilingual Sentence Embeddings with Abstract Meaning Representationcited by 4
- 2022Cross-Lingual Phrase Retrievalcited by 2
Built on by
Citing papers per year: 2025: 1.
1 paper.