mlsanityData to 7 Oct 2026

DeepSeek vs. ChatGPT vs. Claude: A Comparative Study for Scientific Computing and Scientific Machine Learning Tasks

arXiv 2502.17764 ↗First posted 25 Feb 2025cs.LG cs.AIcited by 12

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have emerged as powerful tools for tackling a wide range of problems, including those in scientific computing, particularly in solving partial differential equations (PDEs). However, different models exhibit distinct strengths and preferences, resulting in varying levels of performance. In this paper, we compare the capabilities of the most advanced LLMs--DeepSeek, ChatGPT, and Claude--along with their reasoning-optimized versions in addressing computational challenges. Specifically, we evaluate their proficiency in solving traditional numerical problems in scientific computing as well as leveraging scientific machine learning techniques for PDE-based problems. We designed all our experiments so that a non-trivial decision is required, e.g. defining the proper space of input functions for neural operator learning. Our findings show that reasoning and hybrid-reasoning models consistently and significantly outperform non-reasoning ones in solving challenging problems, with ChatGPT o3-mini-high generally offering the fastest reasoning speed.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 18 of its 47 references that are in the map. First posted on arXiv 25 Feb 2025.

Builds on

18 of 47 references.

References · 18

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 1,965
  3. 2023Using an LLM to Help With Code Understandingcited by 222
  4. 2023Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigationcited by 123
  5. 2024Beyond Functional Correctness: Exploring Hallucinations in LLM-Generated Codecited by 106
  6. 2024MathBench: Evaluating the Theory and Application Proficiency of LLMs with a Hierarchical Mathematics Benchmarkcited by 103
  7. 2023LLM-Assisted Content Analysis: Using Large Language Models to Support Deductive Codingcited by 64
  8. 2021Scientific Machine Learning Benchmarkscited by 60
  9. 2024HoneyComb: A Flexible LLM-Based Agent System for Materials Sciencecited by 56
  10. 2024SciAssess: Benchmarking LLM Proficiency in Scientific Literature Analysiscited by 40
  11. 2024MetaOpenFOAM: an LLM-based multi-agent framework for CFDcited by 39
  12. 2024ChatCite: LLM Agent with Human Workflow Guidance for Comparative Literature Summarycited by 30
6 more
  1. 2024DesignQA: A Multimodal Benchmark for Evaluating Large Language Models' Understanding of Engineering Documentationcited by 30
  2. 2023Deceptive Semantic Shortcuts on Reasoning Chains: How Far Can Models Go without Hallucination?cited by 26
  3. 2024Properties and Challenges of LLM-Generated Explanationscited by 24
  4. 2024U-MATH: A University-Level Benchmark for Evaluating Mathematical Skills in LLMscited by 24
  5. 2023The Impact of ChatGPT and LLMs on Medical Imaging Stakeholders: Perspectives and Use Casescited by 7
  6. 2023GPT vs Human for Scientific Reviews: A Dual Source Review on Applications of ChatGPT in Sciencecited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 10; 2026: 2.

12 papers.

  1. 2025Pre-Trained Policy Discriminators are General Reward Modelscited by 14
  2. 2025Evaluating Mathematical Reasoning Across Large Language Models: A Fine-Grained Approachcited by 8
  3. 2025Text-Trained LLMs Can Zero-Shot Extrapolate PDE Dynamics, Revealing a Three-Stage In-Context Learning Mechanismcited by 7
  4. 2025ATHENA: Agentic Team for Hierarchical Evolutionary Numerical Algorithmscited by 7
  5. 2025SciML Agents: Write the Solver, Not the Solutioncited by 5
  6. 2025CFDLLMBench: A Benchmark Suite for Evaluating Large Language Models in Computational Fluid Dynamicscited by 5
  7. 2026OpInf-LLM: Parametric PDE Solving with LLMs via Operator Inferencecited by 4
  8. 2025Engineering.ai: A Platform for Teams of AI Engineers in Computational Designcited by 3
  9. 2025Re4: Scientific Computing Agent with Rewriting, Resolution, Review and Revisioncited by 2
  10. 2025Comparative Evaluation of ChatGPT and DeepSeek Across Key NLP Tasks: Strengths, Weaknesses, and Domain-Specific Performancecited by 1
  11. 2025AI Needs Physics More Than Physics Needs AIcited by 1
  12. 2026Large language models for partial differential equation workflowscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Showdown of ChatGPT vs DeepSeek in Solving Programming Tasks
  2. 2024LLMs for Mathematical Modeling: Towards Bridging the Gap between Natural and Mathematical Languages
  3. 2024SciAgent: Tool-augmented Language Models for Scientific Reasoning
  4. 2025CodePDE: An Inference Framework for LLM-driven PDE Solver Generation
  5. 2023Have LLMs Advanced Enough? A Challenging Problem Solving Benchmark For Large Language Models
  6. 2025CURIE: Evaluating LLMs On Multitask Scientific Long Context Understanding and Reasoning
  7. 2025Evaluation of LLMs for mathematical problem solving
  8. 2025Challenges and Applications of Large Language Models: A Comparison of GPT and DeepSeek family of models