mlsanityData to 7 Oct 2026

A Conditional Generative Matching Model for Multi-lingual Reply Suggestion

arXiv 2109.07046 ↗First posted 15 Sep 2021cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

We study the problem of multilingual automated reply suggestions (RS) model serving many languages simultaneously. Multilingual models are often challenged by model capacity and severe data distribution skew across languages. While prior works largely focus on monolingual models, we propose Conditional Generative Matching models (CGM), optimized within a Variational Autoencoder framework to address challenges arising from multi-lingual RS. CGM does so with expressive message conditional priors, mixture densities to enhance multi-lingual data representation, latent alignment for language discrimination, and effective variational optimization techniques for training multi-lingual RS. The enhancements result in performance that exceed competitive baselines in relevance (ROUGE score) by more than 10% on average, and 16% for low resource languages. CGM also shows remarkable improvements in diversity (80%) illustrating its expressiveness in representation of multi-lingual data.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 43 of its 58 references that are in the map. First posted on arXiv 15 Sep 2021.

Builds on

43 of 58 references.

References · 43

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  3. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  4. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  5. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,345
  6. 2019Cross-lingual Language Model Pretrainingcited by 1,088
  7. 2015Generating Sentences from a Continuous Spacecited by 607
  8. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 467
  9. 2015Importance Weighted Autoencoderscited by 454
  10. 2016FastText.zip: Compressing text classification modelscited by 331
  11. 2020The Pushshift Reddit Datasetcited by 289
  12. 2020Evaluation of Text Generation: A Surveycited by 275
31 more
  1. 2020InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Trainingcited by 237
  2. 2017Efficient Natural Language Response Suggestion for Smart Replycited by 228
  3. 2017Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoderscited by 222
  4. 2016Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoderscited by 219
  5. 2019Multilingual Universal Sentence Encoder for Semantic Retrievalcited by 192
  6. 2019Cyclical Annealing Schedule: A Simple Approach to Mitigating KL Vanishingcited by 191
  7. 2019Variational Mixture-of-Experts Autoencoders for Multi-Modal Deep Generative Modelscited by 177
  8. 2019To Tune or Not to Tune? Adapting Pretrained Representations to Diverse Taskscited by 167
  9. 2020Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Modelscited by 167
  10. 2019Continual Unsupervised Representation Learningcited by 147
  11. 2015Neural Responding Machine for Short-Text Conversationcited by 124
  12. 2020Optimus: Organizing Sentences via Pre-trained Modeling of a Latent Spacecited by 114
  13. 2019Unsupervised speech representation learning using WaveNet autoencoderscited by 90
  14. 2019What Would Elsa Do? Freezing Layers During Transformer Fine-Tuningcited by 87
  15. 2020Multilingual Offensive Language Identification with Cross-lingual Embeddingscited by 71
  16. 2016Smart Reply: Automated Response Suggestion for Emailcited by 54
  17. 2016Adversarial Deep Averaging Networks for Cross-Lingual Sentiment Classificationcited by 48
  18. 2019Training Neural Response Selection for Task-Oriented Dialogue Systemscited by 48
  19. 2020English Intermediate-Task Training Improves Zero-Shot Cross-Lingual Transfer Toocited by 39
  20. 2016Variational Neural Machine Translationcited by 35
  21. 2020On Negative Interference in Multilingual Models: Findings and A Meta-Learning Treatmentcited by 21
  22. 2017A Conditional Variational Framework for Dialog Generationcited by 16
  23. 2019Diversifying Reply Suggestions using a Matching-Conditional Variational Autoencodercited by 14
  24. 2017Variational Deep Semantic Hashing for Text Documentscited by 12
  25. 2018Auto-Encoding Variational Neural Machine Translationcited by 11
  26. 2018A Collective Variational Autoencoder for Top-N Recommendation with Side Informationcited by 9
  27. 2021A Dataset and Baselines for Multilingual Reply Suggestioncited by 9
  28. 2020Crossing Variational Autoencoders for Answer Retrievalcited by 5
  29. 2019Building a Production Model for Retrieval-Based Chatbotscited by 4
  30. 2020Modelling Latent Skills for Multitask Language Generationcited by 2
  31. 2021Language Scaling for Universal Suggested Replies Modelcited by 2

Built on by

Cited by year

Citing papers per year: 2023: 1.

1 paper.

  1. 2023Model-Based Simulation for Optimising Smart Replycited by 0