# Accelerated materials language processing enabled by GPT

arXiv: https://arxiv.org/abs/2308.09354 · HTML: https://mlsanity.com/paper/2308.09354/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 35 references that are in the map.
- First posted on arXiv 18 Aug 2023.

## Builds on (most cited first, 16 of 16)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty](https://mlsanity.com/paper/1912.02781/) (2019)
- [Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approach](https://mlsanity.com/paper/1909.00161/) (2019)
- [A Survey on Deep Learning for Named Entity Recognition](https://mlsanity.com/paper/1812.09449/) (2018)
- [GPT-NER: Named Entity Recognition via Large Language Models](https://mlsanity.com/paper/2304.10428/) (2023)
- [Evaluating ChatGPT's Information Extraction Capabilities: An Assessment of Performance, Explainability, Calibration, and Faithfulness](https://mlsanity.com/paper/2304.11633/) (2023)
- [A Survey on Recent Advances in Named Entity Recognition from Deep Learning models](https://mlsanity.com/paper/1910.11470/) (2019)
- [MatSciBERT: A Materials Domain Language Model for Text Mining and Information Extraction](https://mlsanity.com/paper/2109.15290/) (2021)
- [Zero-Shot Learning via Semantic Similarity Embedding](https://mlsanity.com/paper/1509.04767/) (2015)
- [ChatGPT Chemistry Assistant for Text Mining and Prediction of MOF Synthesis](https://mlsanity.com/paper/2306.11296/) (2023)
- [A general-purpose material property data extraction pipeline from large polymer corpora using Natural Language Processing](https://mlsanity.com/paper/2209.13136/) (2022)
- [A Survey of Deep Learning Methods for Relation Extraction](https://mlsanity.com/paper/1705.03645/) (2017)
- [Precursor recommendation for inorganic synthesis by machine learning materials similarity from scientific literature](https://mlsanity.com/paper/2302.02303/) (2023)
- [Text Classification using Capsules](https://mlsanity.com/paper/1808.03976/) (2018)

## Abstract (arXiv)

Materials language processing (MLP) is one of the key facilitators of materials science research, as it enables the extraction of structured information from massive materials science literature. Prior works suggested high-performance MLP models for text classification, named entity recognition (NER), and extractive question answering (QA), which require complex model architecture, exhaustive fine-tuning and a large number of human-labelled datasets. In this study, we develop generative pretrained transformer (GPT)-enabled pipelines where the complex architectures of prior MLP models are replaced with strategic designs of prompt engineering. First, we develop a GPT-enabled document classification method for screening relevant documents, achieving comparable accuracy and reliability compared to prior models, with only small dataset. Secondly, for NER task, we design an entity-centric prompts, and learning few-shot of them improved the performance on most of entities in three open datasets. Finally, we develop an GPT-enabled extractive QA model, which provides improved performance and shows the possibility of automatically correcting annotations. While our findings confirm the potential of GPT-enabled MLP models as well as their value in terms of reliability and practicability, our scientific methods and systematic approach are applicable to any materials science domain to accelerate the information extraction of scientific literature.