# GPT-Based Models Meet Simulation: How to Efficiently Use Large-Scale Pre-Trained Language Models Across Simulation Tasks

arXiv: https://arxiv.org/abs/2306.13679 · HTML: https://mlsanity.com/paper/2306.13679/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 67 references that are in the map.
- First posted on arXiv 21 Jun 2023.

## Builds on (most cited first, 18 of 18)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Efficient Transformers: A Survey](https://mlsanity.com/paper/2009.06732/) (2020)
- [A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT](https://mlsanity.com/paper/2302.11382/) (2023)
- [Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model](https://mlsanity.com/paper/2201.11990/) (2022)
- [Predictability and Surprise in Large Generative Models](https://mlsanity.com/paper/2202.07785/) (2022)
- [Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation](https://mlsanity.com/paper/1703.09902/) (2017)
- [A Categorical Archive of ChatGPT Failures](https://mlsanity.com/paper/2302.03494/) (2023)
- [Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Models](https://mlsanity.com/paper/2304.03738/) (2023)
- [Can large language models build causal graphs?](https://mlsanity.com/paper/2303.05279/) (2023)
- [Understanding Causality with Large Language Models: Feasibility and Opportunities](https://mlsanity.com/paper/2304.05524/) (2023)
- [ArguGPT: evaluating, understanding and identifying argumentative essays generated by GPT models](https://mlsanity.com/paper/2304.07666/) (2023)
- [Explainable Automated Debugging via Large Language Model-driven Scientific Debugging](https://mlsanity.com/paper/2304.02195/) (2023)
- [Accessible Visualization via Natural Language Descriptions: A Four-Level Model of Semantic Content](https://mlsanity.com/paper/2110.04406/) (2021)
- [Fully Autonomous Programming with Large Language Models](https://mlsanity.com/paper/2304.10423/) (2023)
- [A Survey of Natural Language Generation](https://mlsanity.com/paper/2112.11739/) (2021)
- [Rich Screen Reader Experiences for Accessible Data Visualization](https://mlsanity.com/paper/2205.04917/) (2022)
- [Innovations in Neural Data-to-text Generation: A Survey](https://mlsanity.com/paper/2207.12571/) (2022)

## Built on by (most cited first, 14 of 14)

- [Shall We Team Up: Exploring Spontaneous Cooperation of Competing LLM Agents](https://mlsanity.com/paper/2402.12327/) (2024)
- [Smart Agent-Based Modeling: On the Use of Large Language Models in Computer Simulations](https://mlsanity.com/paper/2311.06330/) (2023)
- [KAUCUS: Knowledge Augmented User Simulators for Training Language Model Assistants](https://mlsanity.com/paper/2401.16454/) (2024)
- [Large Language Models for Explainable Decisions in Dynamic Digital Twins](https://mlsanity.com/paper/2405.14411/) (2024)
- [Large Language Models for Agent-Based Modelling: Current and possible uses across the modelling cycle](https://mlsanity.com/paper/2507.05723/) (2025)
- [AI Agents as Policymakers in Simulated Epidemics](https://mlsanity.com/paper/2601.04245/) (2026)
- [Exploring the Potential of Conversational AI Support for Agent-Based Social Simulation Model Design](https://mlsanity.com/paper/2405.08032/) (2024)
- [Unlocking the Potential of Past Research: Using Generative AI to Reconstruct Healthcare Simulation Models](https://mlsanity.com/paper/2503.21646/) (2025)
- [Scalable, Symbiotic, AI and Non-AI Agent Based Parallel Discrete Event Simulations](https://mlsanity.com/paper/2505.23846/) (2025)
- [On Integrating Resilience and Human Oversight into LLM-Assisted Modeling Workflows for Digital Twins](https://mlsanity.com/paper/2603.25898/) (2026)
- [Generative Artificial Intelligence (GenAI) to convert images of queuing networks into verifiable simulation models: an open-weight LLM workflow approach](https://mlsanity.com/paper/2607.24259/) (2026)
- [Artificial Intelligence and Modeling & Simulation: An Overview](https://mlsanity.com/paper/2608.00366/) (2026)
- [LLM-based Framework for Generating and Verifying Parallel DEVS Statecharts](https://mlsanity.com/paper/2608.14956/) (2026)
- [Do Personality-Tuned LLMs Make Better Social Agents?](https://mlsanity.com/paper/2609.21857/) (2026)

## Abstract (arXiv)

The disruptive technology provided by large-scale pre-trained language models (LLMs) such as ChatGPT or GPT-4 has received significant attention in several application domains, often with an emphasis on high-level opportunities and concerns. This paper is the first examination regarding the use of LLMs for scientific simulations. We focus on four modeling and simulation tasks, each time assessing the expected benefits and limitations of LLMs while providing practical guidance for modelers regarding the steps involved. The first task is devoted to explaining the structure of a conceptual model to promote the engagement of participants in the modeling process. The second task focuses on summarizing simulation outputs, so that model users can identify a preferred scenario. The third task seeks to broaden accessibility to simulation platforms by conveying the insights of simulation visualizations via text. Finally, the last task evokes the possibility of explaining simulation errors and providing guidance to resolve them.