# Text Summarization Using Large Language Models: A Comparative Study of MPT-7b-instruct, Falcon-7b-instruct, and OpenAI Chat-GPT Models

arXiv: https://arxiv.org/abs/2310.10449 · HTML: https://mlsanity.com/paper/2310.10449/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 1 of its 11 references that are in the map.
- First posted on arXiv 16 Oct 2023.

## Builds on (most cited first, 1 of 1)

- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)

## Built on by (most cited first, 30 of 36)

- [A Comprehensive Survey on Process-Oriented Automatic Text Summarization with Exploration of LLM-Based Methods](https://mlsanity.com/paper/2403.02901/) (2024)
- [User-LLM: Efficient LLM Contextualization with User Embeddings](https://mlsanity.com/paper/2402.13598/) (2024)
- [How Numerical Precision Affects Arithmetical Reasoning Capabilities of LLMs](https://mlsanity.com/paper/2410.13857/) (2024)
- [A Survey of Sustainability in Large Language Models: Applications, Economics, and Challenges](https://mlsanity.com/paper/2412.04782/) (2024)
- [Ontology-Guided Reverse Thinking Makes Large Language Models Stronger on Knowledge Graph Question Answering](https://mlsanity.com/paper/2502.11491/) (2025)
- [Do LLMs Know to Respect Copyright Notice?](https://mlsanity.com/paper/2411.01136/) (2024)
- [Multi-RAG: A Multimodal Retrieval-Augmented Generation System for Adaptive Video Understanding](https://mlsanity.com/paper/2505.23990/) (2025)
- [Anchor-based Large Language Models](https://mlsanity.com/paper/2402.07616/) (2024)
- [ELF-Gym: Evaluating Large Language Models Generated Features for Tabular Prediction](https://mlsanity.com/paper/2410.12865/) (2024)
- [AdaptEval: Evaluating Large Language Models on Domain Adaptation for Text Summarization](https://mlsanity.com/paper/2407.11591/) (2024)
- [Bridging the LLM Accessibility Divide? Performance, Fairness, and Cost of Closed versus Open LLMs for Automated Essay Scoring](https://mlsanity.com/paper/2503.11827/) (2025)
- [An Evaluation of Large Language Models on Text Summarization Tasks Using Prompt Engineering Techniques](https://mlsanity.com/paper/2507.05123/) (2025)
- [Multi-LLM Text Summarization](https://mlsanity.com/paper/2412.15487/) (2024)
- [Fine-Tuned LLMs are "Time Capsules" for Tracking Societal Bias Through Books](https://mlsanity.com/paper/2502.05331/) (2025)
- [Explainable Mapper: Charting LLM Embedding Spaces Using Perturbation-Based Explanation and Verification Agents](https://mlsanity.com/paper/2507.18607/) (2025)
- [EVA-Score: Evaluating Abstractive Long-form Summarization on Informativeness through Extraction and Validation](https://mlsanity.com/paper/2407.04969/) (2024)
- [Evaluating Large Language Models on Financial Report Summarization: An Empirical Study](https://mlsanity.com/paper/2411.06852/) (2024)
- [ArXivBench: When You Should Avoid Using ChatGPT for Academic Writing](https://mlsanity.com/paper/2504.10496/) (2025)
- [Enhancing Abstractive Summarization of Scientific Papers Using Structure Information](https://mlsanity.com/paper/2505.14179/) (2025)
- [To Trade or Not to Trade: An Agentic Approach to Estimating Market Risk Improves Trading Decisions](https://mlsanity.com/paper/2507.08584/) (2025)
- [FastPerson: Enhancing Video Learning through Effective Video Summarization that Preserves Linguistic and Visual Contexts](https://mlsanity.com/paper/2403.17727/) (2024)
- [Decoding BACnet Packets: A Large Language Model Approach for Packet Interpretation](https://mlsanity.com/paper/2407.15428/) (2024)
- [Human Cognition Inspired RAG with Knowledge Graph for Complex Problem Solving](https://mlsanity.com/paper/2503.06567/) (2025)
- [A Large-Scale Empirical Analysis of Custom GPTs' Vulnerabilities in the OpenAI Ecosystem](https://mlsanity.com/paper/2505.08148/) (2025)
- [Do LLMs Understand Collaborative Signals? Diagnosis and Repair](https://mlsanity.com/paper/2505.20730/) (2025)
- [Generating Query-Relevant Document Summaries via Reinforcement Learning](https://mlsanity.com/paper/2508.08404/) (2025)
- [HANRAG: Heuristic Accurate Noise-resistant Retrieval-Augmented Generation for Multi-hop Question Answering](https://mlsanity.com/paper/2509.09713/) (2025)
- [A Regression Framework for Understanding Prompt Component Impact on LLM Performance](https://mlsanity.com/paper/2603.26830/) (2026)
- [HoWToBench: Holistic Evaluation for LLM's Capability in Human-level Writing using Tree of Writing](https://mlsanity.com/paper/2604.19071/) (2026)
- [Enhancing News Summarization with ELearnFit through Efficient In-Context Learning and Efficient Fine-Tuning](https://mlsanity.com/paper/2405.02710/) (2024)

## Abstract (arXiv)

Text summarization is a critical Natural Language Processing (NLP) task with applications ranging from information retrieval to content generation. Leveraging Large Language Models (LLMs) has shown remarkable promise in enhancing summarization techniques. This paper embarks on an exploration of text summarization with a diverse set of LLMs, including MPT-7b-instruct, falcon-7b-instruct, and OpenAI ChatGPT text-davinci-003 models. The experiment was performed with different hyperparameters and evaluated the generated summaries using widely accepted metrics such as the Bilingual Evaluation Understudy (BLEU) Score, Recall-Oriented Understudy for Gisting Evaluation (ROUGE) Score, and Bidirectional Encoder Representations from Transformers (BERT) Score. According to the experiment, text-davinci-003 outperformed the others. This investigation involved two distinct datasets: CNN Daily Mail and XSum. Its primary objective was to provide a comprehensive understanding of the performance of Large Language Models (LLMs) when applied to different datasets. The assessment of these models' effectiveness contributes valuable insights to researchers and practitioners within the NLP domain. This work serves as a resource for those interested in harnessing the potential of LLMs for text summarization and lays the foundation for the development of advanced Generative AI applications aimed at addressing a wide spectrum of business challenges.