mlsanityData to 7 Oct 2026

MQDD: Pre-training of Multimodal Question Duplicity Detection for Software Engineering Domain

arXiv 2203.14093 ↗First posted 26 Mar 2022cs.CL cs.LG cs.PLcited by 0

Not on the map. Why?

Abstract (arXiv)

This work proposes a new pipeline for leveraging data collected on the Stack Overflow website for pre-training a multimodal model for searching duplicates on question answering websites. Our multimodal model is trained on question descriptions and source codes in multiple programming languages. We design two new learning objectives to improve duplicate detection capabilities. The result of this work is a mature, fine-tuned Multimodal Question Duplicity Detection (MQDD) model, ready to be integrated into a Stack Overflow search system, where it can help users find answers for already answered questions. Alongside the MQDD model, we release two datasets related to the software engineering domain. The first Stack Overflow Dataset (SOD) represents a massive corpus of paired questions and answers. The second Stack Overflow Duplicity Dataset (SODD) contains data for training duplicate detection models.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 31 of its 47 references that are in the map. First posted on arXiv 26 Mar 2022.

Builds on

31 of 47 references.

References · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  7. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  8. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  9. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  10. 2020Longformer: The Long-Document Transformercited by 3,362
  11. 2017Billion-scale similarity search with GPUscited by 2,477
  12. 2018Deep contextualized word representationscited by 2,285
19 more
  1. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  2. 2020Reformer: The Efficient Transformercited by 1,658
  3. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
  4. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
  5. 2019VideoBERT: A Joint Model for Video and Language Representation Learningcited by 702
  6. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 638
  7. 2016DeepCoder: Learning to Write Programscited by 145
  8. 2020Structure-Grounded Pretraining for Text-to-SQLcited by 126
  9. 2019Utilizing BERT for Aspect-Based Sentiment Analysis via Constructing Auxiliary Sentencecited by 121
  10. 2020A Survey on Negative Transfercited by 120
  11. 2017Learned in Translation: Contextualized Word Vectorscited by 104
  12. 2020Deep Learning for Source Code Modeling and Generation: Models, Applications and Challengescited by 55
  13. 2019Learning and Evaluating Contextual Embedding of Source Codecited by 33
  14. 2019ATOM: Commit Message Generation Based on Abstract Syntax Tree and Hybrid Rankingcited by 25
  15. 2021NER-BERT: A Pre-trained Model for Low-Resource Entity Taggingcited by 21
  16. 2019A Literature Study of Embeddings on Source Codecited by 18
  17. 2020Neural Code Search Revisited: Enhancing Code Snippet Retrieval through Natural Language Intentcited by 16
  18. 2019Commit2Vec: Learning Distributed Representations of Code Changescited by 12
  19. 2022Code Search based on Context-aware Code Translationcited by 11

Built on by

Not cited yet.