# Transfer Learning for Information Extraction with Limited Data

arXiv: https://arxiv.org/abs/2003.03064 · HTML: https://mlsanity.com/paper/2003.03064/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Mar 2020.

## Built on by (most cited first, 4 of 4)

- [Neural Data-to-Text Generation Based on Small Datasets: Comparing the Added Value of Two Semi-Supervised Learning Approaches on Top of a Large Language Model](https://mlsanity.com/paper/2207.06839/) (2022)
- [When Giant Language Brains Just Aren't Enough! Domain Pizzazz with Knowledge Sparkle Dust](https://mlsanity.com/paper/2305.07230/) (2023)
- [Towards Safer Operations: An Expert-involved Dataset of High-Pressure Gas Incidents for Preventing Future Failures](https://mlsanity.com/paper/2310.12074/) (2023)
- [Radically Lower Data-Labeling Costs for Visually Rich Document Extraction Models](https://mlsanity.com/paper/2210.16391/) (2022)

## Abstract (arXiv)

This paper presents a practical approach to fine-grained information extraction. Through plenty of experiences of authors in practically applying information extraction to business process automation, there can be found a couple of fundamental technical challenges: (i) the availability of labeled data is usually limited and (ii) highly detailed classification is required. The main idea of our proposal is to leverage the concept of transfer learning, which is to reuse the pre-trained model of deep neural networks, with a combination of common statistical classifiers to determine the class of each extracted term. To do that, we first exploit BERT to deal with the limitation of training data in real scenarios, then stack BERT with Convolutional Neural Networks to learn hidden representation for classification. To validate our approach, we applied our model to an actual case of document processing, which is a process of competitive bids for government projects in Japan. We used 100 documents for training and testing and confirmed that the model enables to extract fine-grained named entities with a detailed level of information preciseness specialized in the targeted business process, such as a department name of application receivers.