# DataOps for Societal Intelligence: a Data Pipeline for Labor Market Skills Extraction and Matching

arXiv: https://arxiv.org/abs/2104.01966 · HTML: https://mlsanity.com/paper/2104.01966/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 16 references that are in the map.
- First posted on arXiv 5 Apr 2021.

## Builds on (most cited first, 2 of 2)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [How multilingual is Multilingual BERT?](https://mlsanity.com/paper/1906.01502/) (2019)

## Built on by (most cited first, 13 of 13)

- [SkillSpan: Hard and Soft Skill Extraction from English Job Postings](https://mlsanity.com/paper/2204.12811/) (2022)
- [Design of Negative Sampling Strategies for Distantly Supervised Skill Extraction](https://mlsanity.com/paper/2209.05987/) (2022)
- [ESCOXLM-R: Multilingual Taxonomy-driven Pre-training for the Job Market Domain](https://mlsanity.com/paper/2305.12092/) (2023)
- [Deep Learning-based Computational Job Market Analysis: A Survey on Skill Extraction and Classification from Job Postings](https://mlsanity.com/paper/2402.05617/) (2024)
- [Kompetencer: Fine-grained Skill Classification in Danish Job Postings via Distant Supervision and Transfer Learning](https://mlsanity.com/paper/2205.01381/) (2022)
- [Skill Extraction from Job Postings using Weak Supervision](https://mlsanity.com/paper/2209.08071/) (2022)
- [SkillGPT: a RESTful API service for skill extraction and standardization using a Large Language Model](https://mlsanity.com/paper/2304.11060/) (2023)
- ["FIJO": a French Insurance Soft Skill Detection Dataset](https://mlsanity.com/paper/2204.05208/) (2022)
- [NNOSE: Nearest Neighbor Occupational Skill Extraction](https://mlsanity.com/paper/2401.17092/) (2024)
- [Analysing and Organising Human Communications for AI Fairness-Related Decisions: Use Cases from the Public Sector](https://mlsanity.com/paper/2404.00022/) (2024)
- [Hierarchical Classification of Transversal Skills in Job Ads Based on Sentence Embeddings](https://mlsanity.com/paper/2401.05073/) (2024)
- [Joint Extraction and Classification of Danish Competences for Job Matching](https://mlsanity.com/paper/2410.22103/) (2024)
- [Toward a traceable, explainable, and fairJD/Resume recommendation system](https://mlsanity.com/paper/2202.08960/) (2022)

## Abstract (arXiv)

Big Data analytics supported by AI algorithms can support skills localization and retrieval in the context of a labor market intelligence problem. We formulate and solve this problem through specific DataOps models, blending data sources from administrative and technical partners in several countries into cooperation, creating shared knowledge to support policy and decision-making. We then focus on the critical task of skills extraction from resumes and vacancies featuring state-of-the-art machine learning models. We showcase preliminary results with applied machine learning on real data from the employment agencies of the Netherlands and the Flemish region in Belgium. The final goal is to match these skills to standard ontologies of skills, jobs and occupations.