Utilizing Semantic Textual Similarity for Clinical Survey Data Feature Selection
Not on the map. Why?
Abstract (arXiv)
Survey data can contain a high number of features while having a comparatively low quantity of examples. Machine learning models that attempt to predict outcomes from survey data under these conditions can overfit and result in poor generalizability. One remedy to this issue is feature selection, which attempts to select an optimal subset of features to learn upon. A relatively unexplored source of information in the feature selection process is the usage of textual names of features, which may be semantically indicative of which features are relevant to a target outcome. The relationships between feature names and target names can be evaluated using language models (LMs) to produce semantic textual similarity (STS) scores, which can then be used to select features. We examine the performance using STS to select features directly and in the minimal-redundancy-maximal-relevance (mRMR) algorithm. The performance of STS as a feature selection metric is evaluated against preliminary survey data collected as a part of a clinical study on persistent post-surgical pain (PPSP). The results suggest that features selected with STS can result in higher performance models compared to traditional feature selection algorithms.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 26 of its 63 references that are in the map. First posted on arXiv 19 Aug 2023.
Builds on
26 of 63 references.
References · 26
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
- 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
- 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
- 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
- 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
- 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
- 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
- 2022Large Language Models Encode Clinical Knowledgecited by 1,807
- 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
- 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 998
- 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 996
14 more
- 2019Publicly Available Clinical BERT Embeddingscited by 995
- 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 632
- 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 531
- 2016FastText.zip: Compressing text classification modelscited by 331
- 2022TabLLM: Few-shot Classification of Tabular Data with Large Language Modelscited by 273
- 2022GatorTron: A Large Clinical Language Model to Unlock Patient Information from Unstructured Electronic Health Recordscited by 238
- 2020Byte Pair Encoding is Suboptimal for Language Model Pretrainingcited by 189
- 2023Do We Still Need Clinical Language Models?cited by 97
- 2018MedSTS: A Resource for Clinical Semantic Textual Similaritycited by 39
- 2020Style-transfer and Paraphrase: Looking for a Sensible Semantic Similarity Metriccited by 23
- 2023An Overview on Language Models: Recent Developments and Outlookcited by 22
- 2018Nested cross-validation when selecting classifiers is overzealous for most practical applicationscited by 18
- 2017Benchmarking Relief-Based Feature Selection Methods for Bioinformatics Data Miningcited by 17
- 2017Machine learning approach for early detection of autism by combining questionnaire and home video screeningcited by 7
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.