# NLP for Knowledge Discovery and Information Extraction from Energetics Corpora

arXiv: https://arxiv.org/abs/2402.06964 · HTML: https://mlsanity.com/paper/2402.06964/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 50 references that are in the map.
- First posted on arXiv 10 Feb 2024.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [Galactica: A Large Language Model for Science](https://mlsanity.com/paper/2211.09085/) (2022)
- [ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Prediction](https://mlsanity.com/paper/2010.09885/) (2020)
- [Learning Deep Transformer Models for Machine Translation](https://mlsanity.com/paper/1906.01787/) (2019)
- [SMILES Transformer: Pre-trained Molecular Fingerprint for Low Data Drug Discovery](https://mlsanity.com/paper/1911.04738/) (2019)
- [MatSciBERT: A Materials Domain Language Model for Text Mining and Information Extraction](https://mlsanity.com/paper/2109.15290/) (2021)
- [Mapping the Space of Chemical Reactions Using Attention-Based Neural Networks](https://mlsanity.com/paper/2012.06051/) (2020)
- [Language Models with Transformers](https://mlsanity.com/paper/1904.09408/) (2019)
- [ChatGPT for Programming Numerical Methods](https://mlsanity.com/paper/2303.12093/) (2023)
- [A general-purpose material property data extraction pipeline from large polymer corpora using Natural Language Processing](https://mlsanity.com/paper/2209.13136/) (2022)
- [Deep learning for synthetic microstructure generation in a materials-by-design framework for heterogeneous energetic materials](https://mlsanity.com/paper/2004.04814/) (2020)
- [Looking Through Glass: Knowledge Discovery from Materials Science Literature using Natural Language Processing](https://mlsanity.com/paper/2101.01508/) (2021)
- [Using natural language processing techniques to extract information on the properties and functionalities of energetic materials from large text corpora](https://mlsanity.com/paper/1903.00415/) (2019)
- [Independent Vector Analysis for Data Fusion Prior to Molecular Property Prediction with Machine Learning](https://mlsanity.com/paper/1811.00628/) (2018)
- [Assessing the trade-off between prediction accuracy and interpretability for topic modeling on energetic materials corpora](https://mlsanity.com/paper/2206.00773/) (2022)

## Abstract (arXiv)

We present a demonstration of the utility of NLP for aiding research into energetic materials and associated systems. The NLP method enables machine understanding of textual data, offering an automated route to knowledge discovery and information extraction from energetics text. We apply three established unsupervised NLP models: Latent Dirichlet Allocation, Word2Vec, and the Transformer to a large curated dataset of energetics-related scientific articles. We demonstrate that each NLP algorithm is capable of identifying energetic topics and concepts, generating a language model which aligns with Subject Matter Expert knowledge. Furthermore, we present a document classification pipeline for energetics text. Our classification pipeline achieves 59-76% accuracy depending on the NLP model used, with the highest performing Transformer model rivaling inter-annotator agreement metrics. The NLP approaches studied in this work can identify concepts germane to energetics and therefore hold promise as a tool for accelerating energetics research efforts and energetics material development.