mlsanityData to 14 Oct 2026

Trial2Vec: Zero-Shot Clinical Trial Document Similarity Search using Self-Supervision

arXiv 2206.14719 ↗First posted 29 Jun 2022cs.CL cs.AI cs.LGcited by 18

Not on the map. Why?

Abstract (arXiv)

Clinical trials are essential for drug development but are extremely expensive and time-consuming to conduct. It is beneficial to study similar historical trials when designing a clinical trial. However, lengthy trial documents and lack of labeled data make trial similarity search difficult. We propose a zero-shot clinical trial retrieval method, Trial2Vec, which learns through self-supervision without annotating similar clinical trials. Specifically, the meta-structure of trial documents (e.g., title, eligibility criteria, target disease) along with clinical knowledge (e.g., UMLS knowledge base https://www.nlm.nih.gov/research/umls/index.html) are leveraged to automatically generate contrastive samples. Besides, Trial2Vec encodes trial documents considering meta-structure thus producing compact embeddings aggregating multi-aspect information from the whole document. We show that our method yields medically interpretable embeddings by visualization and it gets a 15% average improvement over the best baselines on precision/recall for trial retrieval, which is evaluated on our labeled 1600 trial pairs. In addition, we prove the pre-trained embeddings benefit the downstream trial outcome prediction task over 240k trials. Software ias available at https://github.com/RyanWangZf/Trial2Vec.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Builds on 53 of its 80 references that are in the map. First posted on arXiv 29 Jun 2022.

Builds on

53 of 80 references.

References · 53

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  7. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  8. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  9. 2020Longformer: The Long-Document Transformercited by 3,413
  10. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  11. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  12. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
38 more
  1. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
  2. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,947
  3. 2015A large annotated corpus for learning natural language inferencecited by 1,847
  4. 2020Big Bird: Transformers for Longer Sequencescited by 1,684
  5. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,380
  6. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
  7. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 1,001
  8. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 849
  9. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 784
  10. 2014Distributed Representations of Sentences and Documentscited by 678
  11. 2019Passage Re-ranking with BERTcited by 645
  12. 2020Overview of the TREC 2019 deep learning trackcited by 286
  13. 2020Pretrained Transformers for Text Ranking: BERT and Beyondcited by 267
  14. 2021Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Samplingcited by 238
  15. 2021ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfercited by 231
  16. 2020Sparse, Dense, and Attentional Representations for Text Retrievalcited by 214
  17. 2021Whitening Sentence Representations for Better Semantics and Faster Retrievalcited by 202
  18. 2020CLEAR: Contrastive Learning for Sentence Representationcited by 186
  19. 2019Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoringcited by 165
  20. 2021Optimizing Dense Retrieval Model Training with Hard Negativescited by 155
  21. 2020Pre-training Tasks for Embedding-based Large-scale Retrievalcited by 153
  22. 2022TransTab: Learning Transferable Tabular Transformers Across Tablescited by 132
  23. 2017A Deep Relevance Matching Model for Ad-hoc Retrievalcited by 126
  24. 2021SelfDoc: Self-Supervised Document Representation Learningcited by 117
  25. 2020An Unsupervised Sentence Embedding Method by Mutual Information Maximizationcited by 91
  26. 2022PromptBERT: Improving BERT Sentence Embeddings with Promptscited by 75
  27. 2021ESimCSE: Enhanced Sample Building Method for Contrastive Learning of Unsupervised Sentence Embeddingcited by 74
  28. 2021WhiteningBERT: An Easy Unsupervised Sentence Embedding Approachcited by 69
  29. 2017Neural Ranking Models with Weak Supervisioncited by 61
  30. 2021Poolingformer: Long Document Modeling with Pooling Attentioncited by 60
  31. 2020Distilling Dense Representations for Ranking using Tightly-Coupled Teacherscited by 52
  32. 2022SNCSE: Contrastive Learning for Unsupervised Sentence Embedding with Soft Negative Samplescited by 39
  33. 2020Universal Sentence Representation Learning with Conditional Masked Language Modelcited by 34
  34. 2020Information Theoretic Counterfactual Learning from Missing-Not-At-Random Feedbackcited by 30
  35. 2020Local Self-Attention over Long Text for Efficient Document Retrievalcited by 18
  36. 2016Learning Latent Vector Spaces for Product Searchcited by 14
  37. 2021Intra-Document Cascading: Learning to Select Passages for Neural Document Rankingcited by 10
  38. 2022Metadata-Induced Contrastive Learning for Zero-Shot Multi-Label Text Classificationcited by 10

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 5; 2024: 5; 2025: 4; 2026: 2.

18 papers.

Sort:

  1. 2024Large Language Models in Drug Discovery and Development: From Disease Mechanisms to Clinical Trialscited by 24
  2. 2022PromptEHR: Conditional Electronic Healthcare Records Generation with Prompt Learningcited by 22
  3. 2023SPOT: Sequential Predictive Modeling of Clinical Trial Outcome with Meta-Learningcited by 13
  4. 2023AutoTrial: Prompting Language Models for Clinical Trial Designcited by 11
  5. 2024Panacea: A foundation model for clinical trial search, summarization, design, and recruitmentcited by 10
  6. 2023Effective Matching of Patients to Clinical Trials using Entity Extraction and Neural Re-rankingcited by 8
  7. 2024A Perspective for Adapting Generalist AI to Specialized Medical AI Applications and Their Challengescited by 8
  8. 2022Artificial Intelligence for In Silico Clinical Trials: A Reviewcited by 6
  9. 2023FRAMM: Fair Ranking with Missing Modalities for Clinical Trial Site Selectioncited by 6
  10. 2023PyTrial: Machine Learning Software and Benchmark for Clinical Trial Applicationscited by 3
  11. 2024CTP-LLM: Clinical Trial Phase Transition Prediction Using Large Language Modelscited by 2
  12. 2025SECRET: Semi-supervised Clinical Trial Document Similarity Searchcited by 2
6 more
  1. 2025Natural Language Processing in Support of Evidence-based Medicine: A Scoping Reviewcited by 2
  2. 2024CTBench: A Comprehensive Benchmark for Evaluating Language Model Capabilities in Clinical Trial Designcited by 1
  3. 2025Deep Learning-based Prediction of Clinical Trial Enrollment with Uncertainty Estimatescited by 0
  4. 2025JEDA: Query-Free Clinical Order Search from Ambient Dialoguescited by 0
  5. 2026POET: Protocol Optimization via Eligibility Tuningcited by 0
  6. 2026DeepImagine: Clinical Trial Outcome Prediction via Stepwise Local Counterfactual Imaginationscited by 0