# Enhancing Cross-lingual Natural Language Inference by Soft Prompting with Multilingual Verbalizer

arXiv: https://arxiv.org/abs/2305.12761 · HTML: https://mlsanity.com/paper/2305.12761/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 39 references that are in the map.
- First posted on arXiv 22 May 2023.

## Builds on (most cited first, 29 of 29)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [Cross-lingual Language Model Pretraining](https://mlsanity.com/paper/1901.07291/) (2019)
- [Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference](https://mlsanity.com/paper/2001.07676/) (2020)
- [XNLI: Evaluating Cross-lingual Sentence Representations](https://mlsanity.com/paper/1809.05053/) (2018)
- [It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners](https://mlsanity.com/paper/2009.07118/) (2020)
- [Word Translation Without Parallel Data](https://mlsanity.com/paper/1710.04087/) (2017)
- [Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond](https://mlsanity.com/paper/1812.10464/) (2018)
- [Learning How to Ask: Querying LMs with Mixtures of Soft Prompts](https://mlsanity.com/paper/2104.06599/) (2021)
- [SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer](https://mlsanity.com/paper/2110.07904/) (2021)
- [CoSDA-ML: Multi-Lingual Code-Switching Data Augmentation for Zero-Shot Cross-Lingual NLP](https://mlsanity.com/paper/2006.06402/) (2020)
- [On Transferability of Prompt Tuning for Natural Language Processing](https://mlsanity.com/paper/2111.06719/) (2021)
- [Discrete and Soft Prompting for Multilingual Models](https://mlsanity.com/paper/2109.03630/) (2021)
- [SelfORE: Self-supervised Relational Feature Learning for Open Relation Extraction](https://mlsanity.com/paper/2004.02438/) (2020)
- [Consistency Regularization for Cross-Lingual Fine-Tuning](https://mlsanity.com/paper/2106.08226/) (2021)
- [Semi-supervised Relation Extraction via Incremental Meta Self-Training](https://mlsanity.com/paper/2010.16410/) (2020)
- [HiURE: Hierarchical Exemplar Contrastive Learning for Unsupervised Relation Extraction](https://mlsanity.com/paper/2205.02225/) (2022)
- [Character-level White-Box Adversarial Attacks against Transformers via Attachable Subwords Substitution](https://mlsanity.com/paper/2210.17004/) (2022)
- [Multilingual Models for Compositional Distributed Semantics](https://mlsanity.com/paper/1404.4641/) (2014)
- [UXLA: A Robust Unsupervised Data Augmentation Framework for Zero-Resource Cross-Lingual NLP](https://mlsanity.com/paper/2004.13240/) (2020)
- [Pair-Level Supervised Contrastive Learning for Natural Language Inference](https://mlsanity.com/paper/2201.10927/) (2022)
- [Syntax-augmented Multilingual BERT for Cross-lingual Transfer](https://mlsanity.com/paper/2106.02134/) (2021)
- [What Makes the Story Forward? Inferring Commonsense Explanations as Prompts for Future Event Generation](https://mlsanity.com/paper/2201.07099/) (2022)
- [A Multi-level Supervised Contrastive Learning Framework for Low-Resource Natural Language Inference](https://mlsanity.com/paper/2205.15550/) (2022)

## Built on by (most cited first, 7 of 7)

- [How do Large Language Models Handle Multilingualism?](https://mlsanity.com/paper/2402.18815/) (2024)
- [Crosslingual Capabilities and Knowledge Barriers in Multilingual Large Language Models](https://mlsanity.com/paper/2406.16135/) (2024)
- [Multi-OphthaLingua: A Multilingual Benchmark for Assessing and Debiasing LLM Ophthalmological QA in LMICs](https://mlsanity.com/paper/2412.14304/) (2024)
- [Order-Level Attention Similarity Across Language Models: A Latent Commonality](https://mlsanity.com/paper/2511.05064/) (2025)
- [Effectively Prompting Small-sized Language Models for Cross-lingual Tasks via Winning Tickets](https://mlsanity.com/paper/2404.01242/) (2024)
- [Cross-Prompt Encoder for Low-Performing Languages](https://mlsanity.com/paper/2508.10352/) (2025)
- [Optimizing Soft Prompt Tuning via Structural Evolution](https://mlsanity.com/paper/2602.16500/) (2026)

## Abstract (arXiv)

Cross-lingual natural language inference is a fundamental problem in cross-lingual language understanding. Many recent works have used prompt learning to address the lack of annotated parallel corpora in XNLI. However, these methods adopt discrete prompting by simply translating the templates to the target language and need external expert knowledge to design the templates. Besides, discrete prompts of human-designed template words are not trainable vectors and can not be migrated to target languages in the inference stage flexibly. In this paper, we propose a novel Soft prompt learning framework with the Multilingual Verbalizer (SoftMV) for XNLI. SoftMV first constructs cloze-style question with soft prompts for the input sample. Then we leverage bilingual dictionaries to generate an augmented multilingual question for the original question. SoftMV adopts a multilingual verbalizer to align the representations of original and augmented multilingual questions into the same semantic space with consistency regularization. Experimental results on XNLI demonstrate that SoftMV can achieve state-of-the-art performance and significantly outperform the previous methods under the few-shot and full-shot cross-lingual transfer settings.