# SkillSpan: Hard and Soft Skill Extraction from English Job Postings

arXiv: https://arxiv.org/abs/2204.12811 · HTML: https://mlsanity.com/paper/2204.12811/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 92 references that are in the map.
- First posted on arXiv 27 Apr 2022.

## Builds on (most cited first, 19 of 19)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books](https://mlsanity.com/paper/1506.06724/) (2015)
- [Publicly Available Clinical BERT Embeddings](https://mlsanity.com/paper/1904.03323/) (2019)
- [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://mlsanity.com/paper/1907.10529/) (2019)
- [BERTweet: A pre-trained language model for English Tweets](https://mlsanity.com/paper/2005.10200/) (2020)
- [Portuguese Named Entity Recognition using BERT-CRF](https://mlsanity.com/paper/1909.10649/) (2019)
- [Few-Shot Question Answering by Pretraining Span Selection](https://mlsanity.com/paper/2101.00438/) (2021)
- [Massive Choice, Ample Tasks (MaChAmp): A Toolkit for Multi-task Learning in NLP](https://mlsanity.com/paper/2005.14672/) (2020)
- [Learning Representations for Soft Skill Matching](https://mlsanity.com/paper/1807.07741/) (2018)
- [Why Comparing Single Performance Scores Does Not Allow to Draw Conclusions About Machine Learning Approaches](https://mlsanity.com/paper/1803.09578/) (2018)
- [DataOps for Societal Intelligence: a Data Pipeline for Labor Market Skills Extraction and Matching](https://mlsanity.com/paper/2104.01966/) (2021)
- [Salience and Market-aware Skill Extraction for Job Targeting](https://mlsanity.com/paper/2005.13094/) (2020)
- [Deep Job Understanding at LinkedIn](https://mlsanity.com/paper/2006.12425/) (2020)
- [De-identification of Privacy-related Entities in Job Postings](https://mlsanity.com/paper/2105.11223/) (2021)

## Built on by (most cited first, 30 of 43)

- [Design of Negative Sampling Strategies for Distantly Supervised Skill Extraction](https://mlsanity.com/paper/2209.05987/) (2022)
- [Large Language Models as Batteries-Included Zero-Shot ESCO Skills Matchers](https://mlsanity.com/paper/2307.03539/) (2023)
- [ESCOXLM-R: Multilingual Taxonomy-driven Pre-training for the Job Market Domain](https://mlsanity.com/paper/2305.12092/) (2023)
- [Deep Learning-based Computational Job Market Analysis: A Survey on Skill Extraction and Classification from Job Postings](https://mlsanity.com/paper/2402.05617/) (2024)
- [Kompetencer: Fine-grained Skill Classification in Danish Job Postings via Distant Supervision and Transfer Learning](https://mlsanity.com/paper/2205.01381/) (2022)
- [Skill Extraction from Job Postings using Weak Supervision](https://mlsanity.com/paper/2209.08071/) (2022)
- [Pre-trained Language Models for Keyphrase Generation: A Thorough Empirical Study](https://mlsanity.com/paper/2212.10233/) (2022)
- [HR-MultiWOZ: A Task Oriented Dialogue (TOD) Dataset for HR LLM Agent](https://mlsanity.com/paper/2402.01018/) (2024)
- [JOBSKAPE: A Framework for Generating Synthetic Job Postings to Enhance Skill Matching](https://mlsanity.com/paper/2402.03242/) (2024)
- [Rethinking Skill Extraction in the Job Market Domain using Large Language Models](https://mlsanity.com/paper/2402.03832/) (2024)
- [Natural Language Processing for Human Resources: A Survey](https://mlsanity.com/paper/2410.16498/) (2024)
- [Entity Linking in the Job Market Domain](https://mlsanity.com/paper/2401.17979/) (2024)
- [Skill-LLM: Repurposing General-Purpose LLMs for Skill Extraction](https://mlsanity.com/paper/2410.12052/) (2024)
- [MELO: An Evaluation Benchmark for Multilingual Entity Linking of Occupations](https://mlsanity.com/paper/2410.08319/) (2024)
- [NNOSE: Nearest Neighbor Occupational Skill Extraction](https://mlsanity.com/paper/2401.17092/) (2024)
- [Efficient Text Encoders for Labor Market Analysis](https://mlsanity.com/paper/2505.24640/) (2025)
- [In-Context Learning for Extreme Multi-Label Classification](https://mlsanity.com/paper/2401.12178/) (2024)
- [Learning Job Title Representation from Job Description Aggregation Network](https://mlsanity.com/paper/2406.08055/) (2024)
- [SkillMatch: Evaluating Self-supervised Learning of Skill Relatedness](https://mlsanity.com/paper/2410.05006/) (2024)
- [Pre-Trained Language Models for Keyphrase Prediction: A Review](https://mlsanity.com/paper/2409.01087/) (2024)
- [Overview of the TalentCLEF 2025: Skill and Job Title Intelligence for Human Capital Management](https://mlsanity.com/paper/2507.13275/) (2025)
- [Protecting Vulnerable Voices: Synthetic Dataset Generation for Self-Disclosure Detection](https://mlsanity.com/paper/2507.22930/) (2025)
- [Enhancing PLM Performance on Labour Market Tasks via Instruction-based Finetuning and Prompt-tuning with Rules](https://mlsanity.com/paper/2308.16770/) (2023)
- [JobResQA: A Benchmark for LLM Machine Reading Comprehension on Multilingual R'esum'es and JDs](https://mlsanity.com/paper/2601.23183/) (2026)
- [Course Recommender Systems Need to Consider the Job Market](https://mlsanity.com/paper/2404.10876/) (2024)
- [Challenges and Opportunities of NLP for HR Applications: A Discussion Paper](https://mlsanity.com/paper/2405.07766/) (2024)
- [Comparative Analysis of Encoder-Based NER and Large Language Models for Skill Extraction from Russian Job Vacancies](https://mlsanity.com/paper/2407.19816/) (2024)
- [Joint Extraction and Classification of Danish Competences for Job Matching](https://mlsanity.com/paper/2410.22103/) (2024)
- [Unified Work Embeddings: Contrastive Learning of a Bidirectional Multi-task Ranker](https://mlsanity.com/paper/2511.07969/) (2025)
- [VietJobs: A Vietnamese Job Advertisement Dataset](https://mlsanity.com/paper/2603.05262/) (2026)

## Abstract (arXiv)

Skill Extraction (SE) is an important and widely-studied task useful to gain insights into labor market dynamics. However, there is a lacuna of datasets and annotation guidelines; available datasets are few and contain crowd-sourced labels on the span-level or labels from a predefined skill inventory. To address this gap, we introduce SKILLSPAN, a novel SE dataset consisting of 14.5K sentences and over 12.5K annotated spans. We release its respective guidelines created over three different sources annotated for hard and soft skills by domain experts. We introduce a BERT baseline (Devlin et al., 2019). To improve upon this baseline, we experiment with language models that are optimized for long spans (Joshi et al., 2020; Beltagy et al., 2020), continuous pre-training on the job posting domain (Han and Eisenstein, 2019; Gururangan et al., 2020), and multi-task learning (Caruana, 1997). Our results show that the domain-adapted models significantly outperform their non-adapted counterparts, and single-task outperforms multi-task learning.