mlsanityData to 7 Oct 2026

UniPSDA: Unsupervised Pseudo Semantic Data Augmentation for Zero-Shot Cross-Lingual Natural Language Understanding

arXiv 2406.16372 ↗First posted 24 Jun 2024cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Cross-lingual representation learning transfers knowledge from resource-rich data to resource-scarce ones to improve the semantic understanding abilities of different languages. However, previous works rely on shallow unsupervised data generated by token surface matching, regardless of the global context-aware semantics of the surrounding text tokens. In this paper, we propose an Unsupervised Pseudo Semantic Data Augmentation (UniPSDA) mechanism for cross-lingual natural language understanding to enrich the training data without human interventions. Specifically, to retrieve the tokens with similar meanings for the semantic data augmentation across different languages, we propose a sequential clustering process in 3 stages: within a single language, across multiple languages of a language family, and across languages from multiple language families. Meanwhile, considering the multi-lingual knowledge infusion with context-aware semantics while alleviating computation burden, we directly replace the key constituents of the sentences with the above-learned multi-lingual family knowledge, viewed as pseudo-semantic. The infusion process is further optimized via three de-biasing techniques without introducing any neural parameters. Extensive experiments demonstrate that our model consistently improves the performance on general zero-shot cross-lingual natural language understanding tasks, including sequence classification, information extraction, and question answering.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 34 of its 62 references that are in the map. First posted on arXiv 24 Jun 2024.

Builds on

34 of 62 references.

References · 34

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  3. 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
  4. 2017Near-linear time approximation algorithms for optimal transport via Sinkhorn iterationcited by 219
  5. 2019Multilingual Universal Sentence Encoder for Semantic Retrievalcited by 192
  6. 2021XLM-T: Multilingual Language Models in Twitter for Sentiment Analysis and Beyondcited by 114
  7. 2017Detecting Visual Relationships with Deep Relational Networkscited by 97
  8. 2020CoSDA-ML: Multi-Lingual Code-Switching Data Augmentation for Zero-Shot Cross-Lingual NLPcited by 79
  9. 2021XLM-E: Cross-lingual Language Model Pre-training via ELECTRAcited by 72
  10. 2019Cross-Lingual Alignment of Contextual Word Embeddings, with Applications to Zero-shot Dependency Parsingcited by 69
  11. 2020Cross-lingual Retrieval for Iterative Self-Supervised Trainingcited by 54
  12. 2020GATE: Graph Attention Transformer Encoder for Cross-lingual Relation and Event Extractioncited by 43
22 more
  1. 2018A Corpus for Multilingual Document Classification in Eight Languagescited by 42
  2. 2020Synthetic Data Augmentation for Zero-Shot Cross-Lingual Question Answeringcited by 41
  3. 2021Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignmentcited by 41
  4. 2022Multilingual Generative Language Models for Zero-Shot Cross-Lingual Event Argument Extractioncited by 40
  5. 2022Sample, Translate, Recombine: Leveraging Audio Alignments for Data Augmentation in End-to-end Speech Translationcited by 27
  6. 2017PPR-FCN: Weakly Supervised Visual Relation Detection via Parallel Pairwise R-FCNcited by 25
  7. 2021Using Optimal Transport as Alignment Objective for fine-tuning Multilingual Contextualized Embeddingscited by 19
  8. 2021XLM-K: Improving Cross-Lingual Language Model Pre-training with Multilingual Knowledgecited by 18
  9. 2018MultiBooked: A Corpus of Basque and Catalan Hotel Reviews Annotated for Aspect-level Sentiment Classificationcited by 17
  10. 2019BiPaR: A Bilingual Parallel Dataset for Multilingual and Cross-lingual Reading Comprehension on Novelscited by 17
  11. 2022A Unified Strategy for Multilingual Grammatical Error Correction with Pre-trained Cross-Lingual Language Modelcited by 16
  12. 2020UXLA: A Robust Unsupervised Data Augmentation Framework for Zero-Resource Cross-Lingual NLPcited by 14
  13. 2021X-METRA-ADA: Cross-lingual Meta-Transfer Learning Adaptation to Natural Language Understanding and Question Answeringcited by 10
  14. 2023Dual-Alignment Pre-training for Cross-lingual Sentence Embeddingcited by 9
  15. 2022Multilingual Pre-training with Language and Task Adaptation for Multilingual Text Style Transfercited by 8
  16. 2018Bilingual Sentiment Embeddings: Joint Projection of Sentiment Across Languagescited by 7
  17. 2021Cross-language Sentence Selection via Data Augmentation and Rationale Trainingcited by 5
  18. 2022Easy Adaptation to Mitigate Gender Bias in Multilingual Text Classificationcited by 5
  19. 2020Data Augmentation with Unsupervised Machine Translation Improves the Structural Similarity of Cross-lingual Word Embeddingscited by 4
  20. 2021Learning from Multiple Noisy Augmented Data Sets for Better Cross-Lingual Spoken Language Understandingcited by 4
  21. 2022Semantically Consistent Data Augmentation for Neural Machine Translation via Conditional Masked Language Modelcited by 3
  22. 2022Detecting Languages Unintelligible to Multilingual Models through Local Structure Probescited by 2

Built on by

Not cited yet.