# Towards Efficient Resume Understanding: A Multi-Granularity Multi-Modal Pre-Training Approach

arXiv: https://arxiv.org/abs/2404.13067 · HTML: https://mlsanity.com/paper/2404.13067/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 33 references that are in the map.
- First posted on arXiv 13 Apr 2024.

## Builds on (most cited first, 14 of 14)

- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://mlsanity.com/paper/1912.13318/) (2019)
- [LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking](https://mlsanity.com/paper/2204.08387/) (2022)
- [DocFormer: End-to-End Transformer for Document Understanding](https://mlsanity.com/paper/2106.11539/) (2021)
- [Large Language Models for Generative Information Extraction: A Survey](https://mlsanity.com/paper/2312.17617/) (2023)
- [LiLT: A Simple yet Effective Language-Independent Layout Transformer for Structured Document Understanding](https://mlsanity.com/paper/2202.13669/) (2022)
- [LayoutXLM: Multimodal Pre-training for Multilingual Visually-rich Document Understanding](https://mlsanity.com/paper/2104.08836/) (2021)
- [Empirical Study of Zero-Shot NER with ChatGPT](https://mlsanity.com/paper/2310.10035/) (2023)
- [Enhancing Person-Job Fit for Talent Recruitment: An Ability-aware Neural Network Approach](https://mlsanity.com/paper/1812.08947/) (2018)
- [A Comprehensive Survey of Artificial Intelligence Techniques for Talent Analytics](https://mlsanity.com/paper/2307.03195/) (2023)
- [Enhancing Question Answering for Enterprise Knowledge Bases using Large Language Models](https://mlsanity.com/paper/2404.08695/) (2024)
- [EZInterviewer: To Improve Job Interview Performance with Mock Interview Generator](https://mlsanity.com/paper/2301.00972/) (2023)
- [Collaborative Company Profiling: Insights from an Employee's Perspective](https://mlsanity.com/paper/1712.02987/) (2017)

## Built on by (most cited first, 3 of 3)

- [RIGL: A Unified Reciprocal Approach for Tracing the Independent and Group Learning Processes](https://mlsanity.com/paper/2406.12465/) (2024)
- [Job-SDF: A Multi-Granularity Dataset for Job Skill Demand Forecasting and Benchmarking](https://mlsanity.com/paper/2406.11920/) (2024)
- [DISCO: A Hierarchical Disentangled Cognitive Diagnosis Framework for Interpretable Job Recommendation](https://mlsanity.com/paper/2410.07671/) (2024)

## Abstract (arXiv)

In the contemporary era of widespread online recruitment, resume understanding has been widely acknowledged as a fundamental and crucial task, which aims to extract structured information from resume documents automatically. Compared to the traditional rule-based approaches, the utilization of recently proposed pre-trained document understanding models can greatly enhance the effectiveness of resume understanding. The present approaches have, however, disregarded the hierarchical relations within the structured information presented in resumes, and have difficulty parsing resumes in an efficient manner. To this end, in this paper, we propose a novel model, namely ERU, to achieve efficient resume understanding. Specifically, we first introduce a layout-aware multi-modal fusion transformer for encoding the segments in the resume with integrated textual, visual, and layout information. Then, we design three self-supervised tasks to pre-train this module via a large number of unlabeled resumes. Next, we fine-tune the model with a multi-granularity sequence labeling task to extract structured information from resumes. Finally, extensive experiments on a real-world dataset clearly demonstrate the effectiveness of ERU.