# Design of Negative Sampling Strategies for Distantly Supervised Skill Extraction

arXiv: https://arxiv.org/abs/2209.05987 · HTML: https://mlsanity.com/paper/2209.05987/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 33 references that are in the map.
- First posted on arXiv 13 Sep 2022.

## Builds on (most cited first, 17 of 17)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Contrastive Learning with Hard Negative Samples](https://mlsanity.com/paper/2010.04592/) (2020)
- [Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approach](https://mlsanity.com/paper/1909.00161/) (2019)
- [RobBERT: a Dutch RoBERTa-based Language Model](https://mlsanity.com/paper/2001.06286/) (2020)
- [Multi-Label Learning from Single Positive Labels](https://mlsanity.com/paper/2106.09708/) (2021)
- [SkillSpan: Hard and Soft Skill Extraction from English Job Postings](https://mlsanity.com/paper/2204.12811/) (2022)
- [JobBERT: Understanding Job Titles through Skills](https://mlsanity.com/paper/2109.09605/) (2021)
- [Extreme Multi-Label Legal Text Classification: A case study in EU Legislation](https://mlsanity.com/paper/1905.10892/) (2019)
- [Acknowledging the Unknown for Multi-label Learning with Single Positive Labels](https://mlsanity.com/paper/2203.16219/) (2022)
- [Learning Representations for Soft Skill Matching](https://mlsanity.com/paper/1807.07741/) (2018)
- [Multi-label Few/Zero-shot Learning with Knowledge Aggregated from Multiple Label Graphs](https://mlsanity.com/paper/2010.07459/) (2020)
- [Extreme Zero-Shot Learning for Extreme Text Classification](https://mlsanity.com/paper/2112.08652/) (2021)
- [DataOps for Societal Intelligence: a Data Pipeline for Labor Market Skills Extraction and Matching](https://mlsanity.com/paper/2104.01966/) (2021)
- [Kompetencer: Fine-grained Skill Classification in Danish Job Postings via Distant Supervision and Transfer Learning](https://mlsanity.com/paper/2205.01381/) (2022)
- ["FIJO": a French Insurance Soft Skill Detection Dataset](https://mlsanity.com/paper/2204.05208/) (2022)
- [Knowledge Base Population using Semantic Label Propagation](https://mlsanity.com/paper/1511.06219/) (2015)

## Built on by (most cited first, 16 of 16)

- [Extreme Multi-Label Skill Extraction Training using Large Language Models](https://mlsanity.com/paper/2307.10778/) (2023)
- [Large Language Models as Batteries-Included Zero-Shot ESCO Skills Matchers](https://mlsanity.com/paper/2307.03539/) (2023)
- [ESCOXLM-R: Multilingual Taxonomy-driven Pre-training for the Job Market Domain](https://mlsanity.com/paper/2305.12092/) (2023)
- [Deep Learning-based Computational Job Market Analysis: A Survey on Skill Extraction and Classification from Job Postings](https://mlsanity.com/paper/2402.05617/) (2024)
- [JOBSKAPE: A Framework for Generating Synthetic Job Postings to Enhance Skill Matching](https://mlsanity.com/paper/2402.03242/) (2024)
- [MELO: An Evaluation Benchmark for Multilingual Entity Linking of Occupations](https://mlsanity.com/paper/2410.08319/) (2024)
- [LELA: an LLM-based Entity Linking Approach with Zero-Shot Domain Adaptation](https://mlsanity.com/paper/2601.05192/) (2026)
- [NNOSE: Nearest Neighbor Occupational Skill Extraction](https://mlsanity.com/paper/2401.17092/) (2024)
- [Efficient Text Encoders for Labor Market Analysis](https://mlsanity.com/paper/2505.24640/) (2025)
- [In-Context Learning for Extreme Multi-Label Classification](https://mlsanity.com/paper/2401.12178/) (2024)
- [A Multi-Stage Framework with Taxonomy-Guided Reasoning for Occupation Classification Using Large Language Models](https://mlsanity.com/paper/2503.12989/) (2025)
- [WorkRB: A Community-Driven Evaluation Framework for AI in the Work Domain](https://mlsanity.com/paper/2604.13055/) (2026)
- [Unified Work Embeddings: Contrastive Learning of a Bidirectional Multi-task Ranker](https://mlsanity.com/paper/2511.07969/) (2025)
- [NLPnorth @ TalentCLEF 2025: Comparing Discriminative, Contrastive, and Prompt-Based Methods for Job Title and Skill Matching](https://mlsanity.com/paper/2506.19058/) (2025)
- [Enhancing Job Matching: Occupation, Skill and Qualification Linking with the ESCO and EQF taxonomies](https://mlsanity.com/paper/2512.03195/) (2025)
- [Leveraging LLMs For Turkish Skill Extraction](https://mlsanity.com/paper/2601.22885/) (2026)

## Abstract (arXiv)

Skills play a central role in the job market and many human resources (HR) processes. In the wake of other digital experiences, today's online job market has candidates expecting to see the right opportunities based on their skill set. Similarly, enterprises increasingly need to use data to guarantee that the skills within their workforce remain future-proof. However, structured information about skills is often missing, and processes building on self- or manager-assessment have shown to struggle with issues around adoption, completeness, and freshness of the resulting data. Extracting skills is a highly challenging task, given the many thousands of possible skill labels mentioned either explicitly or merely described implicitly and the lack of finely annotated training corpora. Previous work on skill extraction overly simplifies the task to an explicit entity detection task or builds on manually annotated training data that would be infeasible if applied to a complete vocabulary of skills. We propose an end-to-end system for skill extraction, based on distant supervision through literal matching. We propose and evaluate several negative sampling strategies, tuned on a small validation dataset, to improve the generalization of skill extraction towards implicitly mentioned skills, despite the lack of such implicit skills in the distantly supervised data. We observe that using the ESCO taxonomy to select negative examples from related skills yields the biggest improvements, and combining three different strategies in one model further increases the performance, up to 8 percentage points in RP@5. We introduce a manually annotated evaluation benchmark for skill extraction based on the ESCO taxonomy, on which we validate our models. We release the benchmark dataset for research purposes to stimulate further research on the task.