mlsanityData to 7 Oct 2026

LLMs for Science: Usage for Code Generation and Data Analysis

arXiv 2311.16733 ↗First posted 28 Nov 2023cs.SE cs.AI cs.CLcited by 34

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have been touted to enable increased productivity in many areas of today's work life. Scientific research as an area of work is no exception: the potential of LLM-based tools to assist in the daily work of scientists has become a highly discussed topic across disciplines. However, we are only at the very onset of this subject of study. It is still unclear how the potential of LLMs will materialise in research practice. With this study, we give first empirical evidence on the use of LLMs in the research process. We have investigated a set of use cases for LLM-based tools in scientific research, and conducted a first study to assess to which degree current tools are helpful. In this paper we report specifically on use cases related to software engineering, such as generating application code and developing scripts for data analytics. While we studied seemingly simple use cases, results across tools differ significantly. Our results highlight the promise of LLM-based tools in general, yet we also observe various issues, particularly regarding the integrity of the output these tools provide.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Builds on 14 of its 26 references that are in the map. First posted on arXiv 28 Nov 2023.

Builds on

14 of 26 references.

References · 14

  1. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  2. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  3. 2021Program Synthesis with Large Language Modelscited by 3,509
  4. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,404
  5. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
  6. 2023A Survey on Evaluation of Large Language Modelscited by 1,227
  7. 2022Galactica: A Large Language Model for Sciencecited by 736
  8. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  9. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
  10. 2022DS-1000: A Natural and Reliable Benchmark for Data Science Code Generationcited by 442
  11. 2022A Systematic Evaluation of Large Language Models of Codecited by 360
  12. 2023The Impact of AI on Developer Productivity: Evidence from GitHub Copilotcited by 264
2 more
  1. 2022The Diminishing Returns of Masked Language Models to Sciencecited by 22
  2. 2023L2CEval: Evaluating Language-to-Code Generation Capabilities of Large Language Modelscited by 22

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 11; 2025: 12; 2026: 11.

34 papers.

Sort:

By area:

  1. 2024A Complete Survey on LLM-based AI Chatbotscited by 71
  2. 2024SciFIBench: Benchmarking Large Multimodal Models for Scientific Figure Interpretationcited by 47
  3. 2024How Far Are We From AGI: Are LLMs All We Need?cited by 31
  4. 2024A Survey on Large Language Model-based Agents for Statistics and Data Sciencecited by 22
  5. 2024Drawing Pandas: A Benchmark for LLMs in Generating Plotting Codecited by 21
  6. 2025Large Language Models for Computer-Aided Design: A Surveycited by 16
  7. 2024On Catastrophic Inheritance of Large Foundation Modelscited by 15
  8. 2024InverseCoder: Self-improving Instruction-Tuned Code LLMs with Inverse-Instructcited by 7
  9. 2025InferA: A Smart Assistant for Cosmological Ensemble Datacited by 6
  10. 2026OpInf-LLM: Parametric PDE Solving with LLMs via Operator Inferencecited by 4
  11. 2024FhGenie: A Custom, Confidentiality-preserving Chat AI for Corporate and Scientific Usecited by 3
  12. 2024Knowing What Not to Do: Leverage Language Model Insights for Action Space Pruning in Multi-agent Reinforcement Learningcited by 3
22 more
  1. 2025Engineering Systems for Data Analysis Using Interactive Structured Inductive Programmingcited by 3
  2. 2026When AI Persuades: Adversarial Explanation Attacks on Human Trust in AI-Assisted Decision Makingcited by 2
  3. 2026CONCUR: Benchmarking LLMs for Concurrent Code Generationcited by 2
  4. 2025Arti-"fickle" Intelligence: Using LLMs as a Tool for Inference in the Political and Social Sciencescited by 1
  5. 2025AutoStreamPipe: LLM Assisted Automatic Generation of Data Stream Processing Pipelinescited by 1
  6. 2026CIAO - Code In Architecture Out - Automated Software Architecture Documentation with Large Language Modelscited by 1
  7. 2024SLEGO: A Collaborative Data Analytics System with LLM Recommender for Diverse Userscited by 0
  8. 2024The why, what, and how of AI-based coding in scientific researchcited by 0
  9. 2025From Scientific Texts to Verifiable Code: Automating the Process with Transformerscited by 0
  10. 2025Navigating Pitfalls: Evaluating LLMs in Machine Learning Programming Educationcited by 0
  11. 2025CrossPL: Evaluating Large Language Models on Cross Programming Language Code Generationcited by 0
  12. 2025Evaluating Generative AI Tools for Personalized Offline Recommendations: A Comparative Studycited by 0
  13. 2025Knowledge-Guided Multi-Agent Framework for Application-Level Software Code Generationcited by 0
  14. 2025AutoICE: Automatically Synthesizing Verifiable C Code via LLM-driven Evolutioncited by 0
  15. 2025CogMCTS: A Novel Cognitive-Guided Monte Carlo Tree Search Framework for Iterative Heuristic Evolution with Large Language Modelscited by 0
  16. 2026Conversational AI for Rapid Scientific Prototyping: A Case Study on ESA's ELOPE Competitioncited by 0
  17. 2026Towards automated data analysis: A guided framework for LLM-based risk estimationcited by 0
  18. 2026AblateCell: A Reproduce-then-Ablate Agent for Virtual Cell Repositoriescited by 0
  19. 2026LLM-driven design of physics-constrained constitutive models: two agents are better than onecited by 0
  20. 2026An Exploratory Case Study of LLM-Assisted Refactoring and Gameplay Feature Generation in an Endless Runner Gamecited by 0
  21. 2026From Overload to Insights: How AI Agents Can Support Scientists in Analyzing Complex Datacited by 0
  22. 2026CURATE: Leveraging LLM Agents to Compose, Catalog, and Deploy Reproducible Workflowscited by 0