mlsanityData to 14 Oct 2026

Learning Semantic Textual Similarity via Topic-informed Discrete Latent Variables

arXiv 2211.03616 ↗First posted 7 Nov 2022cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Recently, discrete latent variable models have received a surge of interest in both Natural Language Processing (NLP) and Computer Vision (CV), attributed to their comparable performance to the continuous counterparts in representation learning, while being more interpretable in their predictions. In this paper, we develop a topic-informed discrete latent variable model for semantic textual similarity, which learns a shared latent space for sentence-pair representation via vector quantization. Compared with previous models limited to local semantic contexts, our model can explore richer semantic information via topic modeling. We further boost the performance of semantic similarity by injecting the quantized representation into a transformer-based language model with a well-designed semantic-driven attention mechanism. We demonstrate, through extensive experiments across various English language datasets, that our model is able to surpass several strong neural baselines in semantic textual similarity tasks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 29 of its 42 references that are in the map. First posted on arXiv 7 Nov 2022.

Builds on

29 of 42 references.

References · 29

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2017Neural Discrete Representation Learningcited by 4,712
  5. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  6. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 658
  7. 2019ERNIE: Enhanced Language Representation with Informative Entitiescited by 528
  8. 2016Enhanced LSTM for Natural Language Inferencecited by 200
  9. 2019PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variablecited by 145
  10. 2017Bilateral Multi-Perspective Matching for Natural Language Sentencescited by 121
  11. 2019Semantics-aware BERT for Language Understandingcited by 112
  12. 2015Neural Variational Inference for Text Processingcited by 111
17 more
  1. 2018Fast Decoding in Sequence Models using Discrete Latent Variablescited by 104
  2. 2019Unsupervised speech representation learning using WaveNet autoencoderscited by 90
  3. 2015Convolutional Neural Network Architectures for Matching Natural Language Sentencescited by 82
  4. 2017Discovering Discrete Latent Topics with Neural Variational Inferencecited by 79
  5. 2014Question Answering with Subgraph Embeddingscited by 58
  6. 2021Topic Modelling Meets Deep Neural Networks: A Surveycited by 51
  7. 2020Neural Topic Model via Optimal Transportcited by 50
  8. 2017Neural Natural Language Inference Models Enhanced with External Knowledgecited by 40
  9. 2015ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairscited by 35
  10. 2019Topic-Aware Neural Keyphrase Generation for Social Media Languagecited by 33
  11. 2018Topic Memory Networks for Short Text Classificationcited by 25
  12. 2021Using Prior Knowledge to Guide BERT's Attention in Semantic Textual Matching Taskscited by 22
  13. 2019SemEval-2017 Task 3: Community Question Answeringcited by 13
  14. 2019SemEval-2016 Task 3: Community Question Answeringcited by 12
  15. 2019SemEval-2015 Task 3: Answer Selection in Community Question Answeringcited by 10
  16. 2020Discrete Latent Variable Representations for Low-Resource Text Classificationcited by 10
  17. 2020Evidence-Aware Inferential Text Generation with Vector Quantised Variational AutoEncodercited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2022Learning to embed semantic similarity for joint image-text retrieval
  2. 2024Advancing Semantic Textual Similarity Modeling: A Regression Framework with Translated ReLU and Smooth K2 Loss
  3. 2024Extracting Sentence Embeddings from Pretrained Transformer Models
  4. 2026Vector Quantized Latent Concepts: A Scalable Alternative to Clustering-Based Concept Discovery
  5. 2025CEMTM: Contextual Embedding-based Multimodal Topic Modeling
  6. 2026Generative Uncertainty as a Self-supervised Signal for Semantic Similarity Learning
  7. 2026Lossy Compressive Text Autoencoders
  8. 2023Deep de Finetti: Recovering Topic Distributions from Large Language Models