# Generative Large Language Models Are All-purpose Text Analytics Engines: Text-to-text Learning Is All Your Need

arXiv: https://arxiv.org/abs/2312.06099 · HTML: https://mlsanity.com/paper/2312.06099/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 48 references that are in the map.
- First posted on arXiv 11 Dec 2023.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://mlsanity.com/paper/1907.10529/) (2019)
- [Neural Architectures for Named Entity Recognition](https://mlsanity.com/paper/1603.01360/) (2016)
- [GatorTron: A Large Clinical Language Model to Unlock Patient Information from Unstructured Electronic Health Records](https://mlsanity.com/paper/2203.03540/) (2022)
- [Large Language Models are Few-Shot Clinical Information Extractors](https://mlsanity.com/paper/2205.12689/) (2022)
- [A Study of Generative Large Language Model for Medical Research and Healthcare](https://mlsanity.com/paper/2305.13523/) (2023)
- [Enhancing Clinical Concept Extraction with Contextual Embeddings](https://mlsanity.com/paper/1902.08691/) (2019)
- [Model Tuning or Prompt Tuning? A Study of Large Language Models for Clinical Concept and Relation Extraction](https://mlsanity.com/paper/2310.06239/) (2023)
- [The 2022 n2c2/UW Shared Task on Extracting Social Determinants of Health](https://mlsanity.com/paper/2301.05571/) (2023)
- [Clinical Concept and Relation Extraction Using Prompt-based Machine Reading Comprehension](https://mlsanity.com/paper/2303.08262/) (2023)
- [Clinical Relation Extraction Using Transformer-based Models](https://mlsanity.com/paper/2107.08957/) (2021)
- [Contextualized Medication Information Extraction Using Transformer-based Deep Learning Architectures](https://mlsanity.com/paper/2303.08259/) (2023)
- [Progress Note Understanding -- Assessment and Plan Reasoning: Overview of the 2022 N2C2 Track 3 Shared Task](https://mlsanity.com/paper/2303.08038/) (2023)

## Built on by (most cited first, 6 of 6)

- [Prompt engineering paradigms for medical applications: scoping review and recommendations for better practices](https://mlsanity.com/paper/2405.01249/) (2024)
- [Automatic Summarization of Doctor-Patient Encounter Dialogues Using Large Language Model through Prompt Tuning](https://mlsanity.com/paper/2403.13089/) (2024)
- [Leveraging Large Language Models to Extract Information on Substance Use Disorder Severity from Clinical Notes: A Zero-shot Learning Approach](https://mlsanity.com/paper/2403.12297/) (2024)
- [Large Language Models for Healthcare Text Classification: A Systematic Review](https://mlsanity.com/paper/2503.01159/) (2025)
- [A Parameter-Efficient Transfer Learning Approach through Multitask Prompt Distillation and Decomposition for Clinical NLP](https://mlsanity.com/paper/2604.06650/) (2026)
- [Detecting HIV-Related Stigma in Clinical Narratives Using Large Language Models](https://mlsanity.com/paper/2604.07717/) (2026)

## Abstract (arXiv)

Objective To solve major clinical natural language processing (NLP) tasks using a unified text-to-text learning architecture based on a generative large language model (LLM) via prompt tuning. Methods We formulated 7 key clinical NLP tasks as text-to-text learning and solved them using one unified generative clinical LLM, GatorTronGPT, developed using GPT-3 architecture and trained with up to 20 billion parameters. We adopted soft prompts (i.e., trainable vectors) with frozen LLM, where the LLM parameters were not updated (i.e., frozen) and only the vectors of soft prompts were updated, known as prompt tuning. We added additional soft prompts as a prefix to the input layer, which were optimized during the prompt tuning. We evaluated the proposed method using 7 clinical NLP tasks and compared them with previous task-specific solutions based on Transformer models. Results and Conclusion The proposed approach achieved state-of-the-art performance for 5 out of 7 major clinical NLP tasks using one unified generative LLM. Our approach outperformed previous task-specific transformer models by ~3% for concept extraction and 7% for relation extraction applied to social determinants of health, 3.4% for clinical concept normalization, 3.4~10% for clinical abbreviation disambiguation, and 5.5~9% for natural language inference. Our approach also outperformed a previously developed prompt-based machine reading comprehension (MRC) model, GatorTron-MRC, for clinical concept and relation extraction. The proposed approach can deliver the ``one model for all`` promise from training to deployment using a unified generative LLM.