# Efficient Text Encoders for Labor Market Analysis

arXiv: https://arxiv.org/abs/2505.24640 · HTML: https://mlsanity.com/paper/2505.24640/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 37 references that are in the map.
- First posted on arXiv 30 May 2025.

## Builds on (most cited first, 22 of 22)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020)
- [Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup](https://mlsanity.com/paper/2101.06983/) (2021)
- [Fast Multi-Resolution Transformer Fine-tuning for Extreme Multi-label Text Classification](https://mlsanity.com/paper/2110.00685/) (2021)
- [LightXML: Transformer with Dynamic Negative Sampling for High-Performance Extreme Multi-label Text Classification](https://mlsanity.com/paper/2101.03305/) (2021)
- [SkillSpan: Hard and Soft Skill Extraction from English Job Postings](https://mlsanity.com/paper/2204.12811/) (2022)
- [DeepXML: A Deep Extreme Multi-Label Learning Framework Applied to Short Text Documents](https://mlsanity.com/paper/2111.06685/) (2021)
- [JobBERT: Understanding Job Titles through Skills](https://mlsanity.com/paper/2109.09605/) (2021)
- [Extreme Multi-Label Legal Text Classification: A case study in EU Legislation](https://mlsanity.com/paper/1905.10892/) (2019)
- [Learning Representations for Soft Skill Matching](https://mlsanity.com/paper/1807.07741/) (2018)
- [Extreme Multi-Label Skill Extraction Training using Large Language Models](https://mlsanity.com/paper/2307.10778/) (2023)
- [Design of Negative Sampling Strategies for Distantly Supervised Skill Extraction](https://mlsanity.com/paper/2209.05987/) (2022)
- [Large Language Models as Batteries-Included Zero-Shot ESCO Skills Matchers](https://mlsanity.com/paper/2307.03539/) (2023)
- [De-identification of Privacy-related Entities in Job Postings](https://mlsanity.com/paper/2105.11223/) (2021)
- [Learning Job Titles Similarity from Noisy Skill Labels](https://mlsanity.com/paper/2207.00494/) (2022)
- [VacancySBERT: the approach for representation of titles and skills for semantic similarity search in the recruitment domain](https://mlsanity.com/paper/2307.16638/) (2023)
- [Skill-LLM: Repurposing General-Purpose LLMs for Skill Extraction](https://mlsanity.com/paper/2410.12052/) (2024)
- [NNOSE: Nearest Neighbor Occupational Skill Extraction](https://mlsanity.com/paper/2401.17092/) (2024)
- [In-Context Learning for Extreme Multi-Label Classification](https://mlsanity.com/paper/2401.12178/) (2024)
- [Learning Job Title Representation from Job Description Aggregation Network](https://mlsanity.com/paper/2406.08055/) (2024)
- [Diversity Subsampling: Custom Subsamples from Large Data Sets](https://mlsanity.com/paper/2206.10812/) (2022)

## Built on by (most cited first, 5 of 5)

- [WorkRB: A Community-Driven Evaluation Framework for AI in the Work Domain](https://mlsanity.com/paper/2604.13055/) (2026)
- [Diagnosable ColBERT: Debugging Late-Interaction Retrieval Models Using a Learned Latent Space as Reference](https://mlsanity.com/paper/2604.19566/) (2026)
- [Unified Work Embeddings: Contrastive Learning of a Bidirectional Multi-task Ranker](https://mlsanity.com/paper/2511.07969/) (2025)
- [Leveraging LLMs For Turkish Skill Extraction](https://mlsanity.com/paper/2601.22885/) (2026)
- [An Agentic Hybrid Top-Down and Bottom-Up Approach to Knowledge Graph Generation](https://mlsanity.com/paper/2608.07023/) (2026)

## Abstract (arXiv)

Labor market analysis relies on extracting insights from job advertisements, which provide valuable yet unstructured information on job titles and corresponding skill requirements. While state-of-the-art methods for skill extraction achieve strong performance, they depend on large language models (LLMs), which are computationally expensive and slow. In this paper, we propose ConTeXT-match, a novel contrastive learning approach with token-level attention that is well-suited for the extreme multi-label classification task of skill classification. ConTeXT-match significantly improves skill extraction efficiency and performance, achieving state-of-the-art results with a lightweight bi-encoder model. To support robust evaluation, we introduce Skill-XL, a new benchmark with exhaustive, sentence-level skill annotations that explicitly address the redundancy in the large label space. Finally, we present JobBERT V2, an improved job title normalization model that leverages extracted skills to produce high-quality job title representations. Experiments demonstrate that our models are efficient, accurate, and scalable, making them ideal for large-scale, real-time labor market analysis.