# Overview of the SV-Ident 2022 Shared Task on Survey Variable Identification in Social Science Publications

arXiv: https://arxiv.org/abs/2209.09062 · HTML: https://mlsanity.com/paper/2209.09062/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 32 references that are in the map.
- First posted on arXiv 19 Sep 2022.

## Builds on (most cited first, 10 of 10)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation](https://mlsanity.com/paper/2004.09813/) (2020)
- [Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models](https://mlsanity.com/paper/2108.08877/) (2021)
- [SPARTA: Efficient Open-Domain Question Answering via Sparse Transformer Matching Retrieval](https://mlsanity.com/paper/2009.13013/) (2020)
- [MedSTS: A Resource for Clinical Semantic Textual Similarity](https://mlsanity.com/paper/1808.09397/) (2018)
- [SsciBERT: A Pre-trained Language Model for Social Science Texts](https://mlsanity.com/paper/2206.04510/) (2022)
- [CORD19STS: COVID-19 Semantic Textual Similarity Dataset](https://mlsanity.com/paper/2007.02461/) (2020)
- [Towards Automated Survey Variable Search and Summarization in Social Science Publications](https://mlsanity.com/paper/2209.06804/) (2022)

## Built on by (most cited first, 3 of 3)

- [Evaluating end-to-end entity linking on domain-specific knowledge bases: Learning about ancient technologies from museum collections](https://mlsanity.com/paper/2305.14588/) (2023)
- [VADIS -- a VAriable Detection, Interlinking and Summarization system](https://mlsanity.com/paper/2312.13423/) (2023)
- [Enriching Social Science Research via Survey Item Linking](https://mlsanity.com/paper/2412.15831/) (2024)

## Abstract (arXiv)

In this paper, we provide an overview of the SV-Ident shared task as part of the 3rd Workshop on Scholarly Document Processing (SDP) at COLING 2022. In the shared task, participants were provided with a sentence and a vocabulary of variables, and asked to identify which variables, if any, are mentioned in individual sentences from scholarly documents in full text. Two teams made a total of 9 submissions to the shared task leaderboard. While none of the teams improve on the baseline systems, we still draw insights from their submissions. Furthermore, we provide a detailed evaluation. Data and baselines for our shared task are freely available at https://github.com/vadis-project/sv-ident