# Beyond original Research Articles Categorization via NLP

arXiv: https://arxiv.org/abs/2309.07020 · HTML: https://mlsanity.com/paper/2309.07020/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 25 references that are in the map.
- First posted on arXiv 13 Sep 2023.

## Builds on (most cited first, 11 of 11)

- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [Pre-trained Models for Natural Language Processing: A Survey](https://mlsanity.com/paper/2003.08271/) (2020)
- [How to Fine-Tune BERT for Text Classification?](https://mlsanity.com/paper/1905.05583/) (2019)
- [Cross-lingual Contextualized Topic Models with Zero-shot Learning](https://mlsanity.com/paper/2004.07737/) (2020)
- [The CSO Classifier: Ontology-Driven Detection of Research Topics in Scholarly Articles](https://mlsanity.com/paper/2104.00948/) (2021)
- [Using Deep Learning for Title-Based Semantic Subject Indexing to Reach Competitive Performance to Full-Text](https://mlsanity.com/paper/1801.06717/) (2018)
- [Large Scale Subject Category Classification of Scholarly Papers with Deep Attentive Neural Networks](https://mlsanity.com/paper/2007.13826/) (2020)
- [Using Titles vs. Full-text as Source for Automated Semantic Document Annotation](https://mlsanity.com/paper/1705.05311/) (2017)
- [Profiling vs. Time vs. Content: What does Matter for Top-k Publication Recommendation based on Twitter Profiles? - An Extended Technical Report](https://mlsanity.com/paper/1603.07016/) (2016)

## Abstract (arXiv)

This work proposes a novel approach to text categorization -- for unknown categories -- in the context of scientific literature, using Natural Language Processing techniques. The study leverages the power of pre-trained language models, specifically SciBERT, to extract meaningful representations of abstracts from the ArXiv dataset. Text categorization is performed using the K-Means algorithm, and the optimal number of clusters is determined based on the Silhouette score. The results demonstrate that the proposed approach captures subject information more effectively than the traditional arXiv labeling system, leading to improved text categorization. The approach offers potential for better navigation and recommendation systems in the rapidly growing landscape of scientific research literature.