# Capturing research literature attitude towards Sustainable Development Goals: an LLM-based topic modeling approach

arXiv: https://arxiv.org/abs/2411.02943 · HTML: https://mlsanity.com/paper/2411.02943/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 80 references that are in the map.
- First posted on arXiv 5 Nov 2024.

## Builds on (most cited first, 22 of 22)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [MTEB: Massive Text Embedding Benchmark](https://mlsanity.com/paper/2210.07316/) (2022)
- [SPECTER: Document-level Representation Learning using Citation-informed Transformers](https://mlsanity.com/paper/2004.07180/) (2020)
- [Topic Modeling in Embedding Spaces](https://mlsanity.com/paper/1907.04907/) (2019)
- [Bias and Unfairness in Information Retrieval Systems: New Challenges in the LLM Era](https://mlsanity.com/paper/2404.11457/) (2024)
- [Tired of Topic Models? Clusters of Pretrained Word Embeddings Make for Fast and Good Topics too!](https://mlsanity.com/paper/2004.14914/) (2020)
- [Topic Modeling with Contextualized Word Representation Clusters](https://mlsanity.com/paper/2010.12626/) (2020)
- [Large Language Models Offer an Alternative to the Traditional Approach of Topic Modelling](https://mlsanity.com/paper/2403.16248/) (2024)
- [Topic Discovery via Latent Space Clustering of Pretrained Language Model Representations](https://mlsanity.com/paper/2202.04582/) (2022)
- [Text Clustering with Large Language Model Embeddings](https://mlsanity.com/paper/2403.15112/) (2024)
- [Prompting Large Language Models for Topic Modeling](https://mlsanity.com/paper/2312.09693/) (2023)
- [Mixing Dirichlet Topic Models and Word Embeddings to Make lda2vec](https://mlsanity.com/paper/1605.02019/) (2016)
- [The Journal Coverage of Web of Science and Scopus: a Comparative Analysis](https://mlsanity.com/paper/1511.08096/) (2015)
- [Analysis of the influence of political polarization in the vaccination stance: the Brazilian COVID-19 scenario](https://mlsanity.com/paper/2110.03382/) (2021)
- [Artificial intelligence for Sustainable Energy: A Contextual Topic Modeling and Content Analysis](https://mlsanity.com/paper/2110.00828/) (2021)
- [Exploring the evolution of research topics during the COVID-19 pandemic](https://mlsanity.com/paper/2310.03928/) (2023)
- [Leveraging Artificial Intelligence Technology for Mapping Research to Sustainable Development Goals: A Case Study](https://mlsanity.com/paper/2311.16162/) (2023)
- [Predicting Sustainable Development Goals Using Course Descriptions -- from LLMs to Conventional Foundation Models](https://mlsanity.com/paper/2402.16420/) (2024)

## Built on by (most cited first, 1 of 1)

- [MALTopic: Multi-Agent LLM Topic Modeling Framework](https://mlsanity.com/paper/2601.15299/) (2026)

## Abstract (arXiv)

The world is facing a multitude of challenges that hinder the development of human civilization and the well-being of humanity on the planet. The Sustainable Development Goals (SDGs) were formulated by the United Nations in 2015 to address these global challenges by 2030. Natural language processing techniques can help uncover discussions on SDGs within research literature. We propose a completely automated pipeline to 1) fetch content from the Scopus database and prepare datasets dedicated to five groups of SDGs; 2) perform topic modeling, a statistical technique used to identify topics in large collections of textual data; and 3) enable topic exploration through keywords-based search and topic frequency time series extraction. For topic modeling, we leverage the stack of BERTopic scaled up to be applied on large corpora of textual documents (we find hundreds of topics on hundreds of thousands of documents), introducing i) a novel LLM-based embeddings computation for representing scientific abstracts in the continuous space and ii) a hyperparameter optimizer to efficiently find the best configuration for any new big datasets. We additionally produce the visualization of results on interactive dashboards reporting topics' temporal evolution. Results are made inspectable and explorable, contributing to the interpretability of the topic modeling process. Our proposed LLM-based topic modeling pipeline for big-text datasets allows users to capture insights on the evolution of the attitude toward SDGs within scientific abstracts in the 2006-2023 time span. All the results are reproducible by using our system; the workflow can be generalized to be applied at any point in time to any big corpus of textual documents.