# AI Insights: A Case Study on Utilizing ChatGPT Intelligence for Research Paper Analysis

arXiv: https://arxiv.org/abs/2403.03293 · HTML: https://mlsanity.com/paper/2403.03293/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 31 references that are in the map.
- First posted on arXiv 5 Mar 2024.

## Builds on (most cited first, 3 of 3)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- ["I think this is the most disruptive technology": Exploring Sentiments of ChatGPT Early Adopters using Twitter Data](https://mlsanity.com/paper/2212.05856/) (2022)
- [Artificial intelligence to automate the systematic review of scientific literature](https://mlsanity.com/paper/2401.10917/) (2024)

## Built on by (most cited first, 2 of 2)

- [Highlighting Case Studies in LLM Literature Review of Interdisciplinary System Science](https://mlsanity.com/paper/2503.16515/) (2025)
- [A Transformer-based Autoregressive Decoder Architecture for Hierarchical Text Classification](https://mlsanity.com/paper/2501.13598/) (2025)

## Abstract (arXiv)

This paper discusses the effectiveness of leveraging Chatbot: Generative Pre-trained Transformer (ChatGPT) versions 3.5 and 4 for analyzing research papers for effective writing of scientific literature surveys. The study selected the textitApplication of Artificial Intelligence in Breast Cancer Treatment as the research topic. Research papers related to this topic were collected from three major publication databases Google Scholar, Pubmed, and Scopus. ChatGPT models were used to identify the category, scope, and relevant information from the research papers for automatic identification of relevant papers related to Breast Cancer Treatment (BCT), organization of papers according to scope, and identification of key information for survey paper writing. Evaluations performed using ground truth data annotated using subject experts reveal, that GPT-4 achieves 77.3% accuracy in identifying the research paper categories and 50% of the papers were correctly identified by GPT-4 for their scopes. Further, the results demonstrate that GPT-4 can generate reasons for its decisions with an average of 27% new words, and 67% of the reasons given by the model were completely agreeable to the subject experts.