# Analyzing Scientific Publications using Domain-Specific Word Embedding and Topic Modelling

arXiv: https://arxiv.org/abs/2112.12940 · HTML: https://mlsanity.com/paper/2112.12940/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 51 references that are in the map.
- First posted on arXiv 24 Dec 2021.

## Builds on (most cited first, 15 of 15)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Learning to Extract Semantic Structure from Documents Using Multimodal Fully Convolutional Neural Network](https://mlsanity.com/paper/1706.02337/) (2017)
- [Word Embeddings: A Survey](https://mlsanity.com/paper/1901.09069/) (2019)
- [Re-translation versus Streaming for Simultaneous Translation](https://mlsanity.com/paper/2004.03643/) (2020)
- [CrisisBERT: a Robust Transformer for Crisis Classification and Contextual Crisis Embedding](https://mlsanity.com/paper/2005.06627/) (2020)
- [Exploratory Analysis of Covid-19 Tweets using Topic Modeling, UMAP, and DiGraphs](https://mlsanity.com/paper/2005.03082/) (2020)
- [Topic Modeling Genre: An Exploration of French Classical and Enlightenment Drama](https://mlsanity.com/paper/2103.13019/) (2021)
- [EPIC30M: An Epidemics Corpus Of Over 30 Million Relevant Tweets](https://mlsanity.com/paper/2006.08369/) (2020)
- [OCR++: A Robust Framework For Information Extraction from Scholarly Articles](https://mlsanity.com/paper/1609.06423/) (2016)
- [Understanding Public Sentiments, Opinions and Topics about COVID-19 using Twitter](https://mlsanity.com/paper/2012.03039/) (2020)
- [TweetCOVID: A System for Analyzing Public Sentiments and Discussions about COVID-19 via Twitter Activities](https://mlsanity.com/paper/2103.01472/) (2021)
- [Real-time Spatio-temporal Event Detection on Geotagged Social Media](https://mlsanity.com/paper/2106.13121/) (2021)

## Built on by (most cited first, 1 of 1)

- [Spatial-Temporal Graph Representation Learning for Tactical Networks Future State Prediction](https://mlsanity.com/paper/2403.13872/) (2024)

## Abstract (arXiv)

The scientific world is changing at a rapid pace, with new technology being developed and new trends being set at an increasing frequency. This paper presents a framework for conducting scientific analyses of academic publications, which is crucial to monitor research trends and identify potential innovations. This framework adopts and combines various techniques of Natural Language Processing, such as word embedding and topic modelling. Word embedding is used to capture semantic meanings of domain-specific words. We propose two novel scientific publication embedding, i.e., PUB-G and PUB-W, which are capable of learning semantic meanings of general as well as domain-specific words in various research fields. Thereafter, topic modelling is used to identify clusters of research topics within these larger research fields. We curated a publication dataset consisting of two conferences and two journals from 1995 to 2020 from two research domains. Experimental results show that our PUB-G and PUB-W embeddings are superior in comparison to other baseline embeddings by a margin of ~0.18-1.03 based on topic coherence.