mlsanityData to 14 Oct 2026

Learning Agent-based Modeling with LLM Companions: Experiences of Novices and Experts Using ChatGPT & NetLogo Chat

arXiv 2401.17163 ↗First posted 30 Jan 2024cs.HCcited by 23

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Large Language Models (LLMs) have the potential to fundamentally change the way people engage in computer programming. Agent-based modeling (ABM) has become ubiquitous in natural and social sciences and education, yet no prior studies have explored the potential of LLMs to assist it. We designed NetLogo Chat to support the learning and practice of NetLogo, a programming language for ABM. To understand how users perceive, use, and need LLM-based interfaces, we interviewed 30 participants from global academia, industry, and graduate schools. Experts reported more perceived benefits than novices and were more inclined to adopt LLMs in their workflow. We found significant differences between experts and novices in their perceptions, behaviors, and needs for human-AI collaboration. We surfaced a knowledge gap between experts and novices as a possible reason for the benefit gap. We identified guidance, personalization, and integration as major needs for LLM-based interfaces to support the programming of ABM.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 25 of its 116 references that are in the map. First posted on arXiv 30 Jan 2024.

Builds on

25 of 116 references.

References · 25

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  5. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  6. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
  7. 2023The Impact of AI on Developer Productivity: Evidence from GitHub Copilotcited by 274
  8. 2022Grounded Copilot: How Programmers Interact with Code-Generating Modelscited by 218
  9. 2022Do Users Write More Insecure Code with AI Assistants?cited by 192
  10. 2022GitHub Copilot AI pair programmer: Asset or Liability?cited by 152
  11. 2023The Programmer's Assistant: Conversational Interaction with a Large Language Model for Software Developmentcited by 96
  12. 2020Low-resource Languages: A Review of Past Work and Future Challengescited by 91
13 more
  1. 2023Studying the effect of AI Code Generators on Supporting Novice Learners in Introductory Programmingcited by 89
  2. 2023Is ChatGPT the Ultimate Programming Assistant -- How far is it?cited by 76
  3. 2022Repair Is Nearly Generation: Multilingual Program Repair with LLMscited by 68
  4. 2023Grammar Prompting for Domain-Specific Language Generation with Large Language Modelscited by 58
  5. 2019What Makes a Good Conversation? Challenges in Designing Truly Conversational Agentscited by 42
  6. 2023Thrilled by Your Progress! Large Language Models (GPT-4) No Longer Struggle to Pass Assessments in Higher Education Programming Coursescited by 39
  7. 2022Automatic Code Documentation Generation Using GPT-3cited by 37
  8. 2022On the Transferability of Pre-trained Language Models for Low-Resource Programming Languagescited by 30
  9. 2023On the Design of AI-powered Code Assistants for Notebookscited by 28
  10. 2023Beyond Generating Code: Evaluating GPT on a Data Visualization Coursecited by 16
  11. 2023The potential of LLMs for coding with low-resource and domain-specific programming languagescited by 11
  12. 2023MyCrunchGPT: A chatGPT assisted framework for scientific machine learningcited by 8
  13. 2022MultiCoder: Multi-Programming-Lingual Pre-Training for Low-Resource Code Completioncited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 6; 2025: 11; 2026: 6.

23 papers.

Sort:

By area:

  1. 2024Using AI-Based Coding Assistants in Practice: State of Affairs, Perceptions, and Ways Forwardcited by 45
  2. 2024MathVC: An LLM-Simulated Multi-Character Virtual Classroom for Mathematics Educationcited by 37
  3. 2024What Should We Engineer in Prompts? Training Humans in Requirement-Driven LLM Usecited by 24
  4. 2025Understanding the LLM-ification of CHI: Unpacking the Impact of LLMs at CHI through a Systematic Literature Reviewcited by 16
  5. 2024Large Language Model-Enabled Multi-Agent Manufacturing Systemscited by 14
  6. 2024Beyond the Hype: A Comprehensive Review of Current Trends in Generative AI Research, Teaching Practices, and Toolscited by 14
  7. 2025Can Large Language Models Identify Implicit Suicidal Ideation? An Empirical Evaluationcited by 13
  8. 2024Broadening Access to Simulations for End-Users via Large Language Models: Challenges and Opportunitiescited by 4
  9. 2025Large Language Models for Agent-Based Modelling: Current and possible uses across the modelling cyclecited by 4
  10. 2025Carbon and Silicon, Coexist or Compete? A Survey on Human-AI Interactions in Agent-based Modeling and Simulationcited by 3
  11. 2025Novice Developers' Perspectives on Adopting LLMs for Software Development: A Systematic Literature Reviewcited by 3
  12. 2025From Consumption to Collaboration: Measuring Interaction Patterns to Augment Human Cognition in Open-Ended Taskscited by 3
11 more
  1. 2025"Would You Want an AI Tutor?" Understanding Stakeholder Perceptions of LLM-based Systems in the Classroomcited by 2
  2. 2025Using (Not-so) Large Language Models to Generate Simulation Models in a Formal DSL: A Study on Reaction Networkscited by 1
  3. 2025Unveiling the Learning Mind of Language Models: A Cognitive Framework and Empirical Studycited by 1
  4. 2025Is Passive Expertise-Based Personalization Enough? A Case Study in AI-Assisted Test-Takingcited by 1
  5. 2026False Friends in the Shell: Unveiling the Emoticon Semantic Confusion in Large Language Modelscited by 1
  6. 2026On Integrating Resilience and Human Oversight into LLM-Assisted Modeling Workflows for Digital Twinscited by 1
  7. 2026Priming, Path-dependence, and Plasticity: Understanding the molding of user-LLM interaction and its implications from (many) chat logs in the wildcited by 1
  8. 2025Feasibility of AI-Assisted Programming for End-User Developmentcited by 0
  9. 2026Upskilling with Generative AI: Practices and Challenges for Freelance Knowledge Workerscited by 0
  10. 2026Artificial Intelligence and Modeling & Simulation: An Overviewcited by 0
  11. 2026Clarify User Expertise: Towards Proactive Conversational Agents Tailoring Responses to User Proficiencycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023ChatLogo: A Large Language Model-Driven Hybrid Natural-Programming Language Interface for Agent-based Modeling and Programming
  2. 2023A Survey on Large Language Model based Autonomous Agents
  3. 2026A Scoping Review of Large Language Model-Based Pedagogical Agents
  4. 2023The Rise and Potential of Large Language Model Based Agents: A Survey
  5. 2024Large Language Model-Based Agents for Software Engineering: A Survey
  6. 2025LLM Agents for Education: Advances and Applications
  7. 2023AgentBench: Evaluating LLMs as Agents
  8. 2025LLM-Based Human-Agent Collaboration and Interaction Systems: A Survey