mlsanityData to 14 Oct 2026

Capabilities of Large Language Models in Control Engineering: A Benchmark Study on GPT-4, Claude 3 Opus, and Gemini 1.0 Ultra

arXiv 2404.03647 ↗First posted 4 Apr 2024math.OC cs.AI cs.LGcited by 37

Not on the map. Why?

Abstract (arXiv)

In this paper, we explore the capabilities of state-of-the-art large language models (LLMs) such as GPT-4, Claude 3 Opus, and Gemini 1.0 Ultra in solving undergraduate-level control problems. Controls provides an interesting case study for LLM reasoning due to its combination of mathematical theory and engineering design. We introduce ControlBench, a benchmark dataset tailored to reflect the breadth, depth, and complexity of classical control design. We use this dataset to study and evaluate the problem-solving abilities of these LLMs in the context of control engineering. We present evaluations conducted by a panel of human experts, providing insights into the accuracy, reasoning, and explanatory prowess of LLMs in control engineering. Our analysis reveals the strengths and limitations of each LLM in the context of classical control, and our results imply that Claude 3 Opus has become the state-of-the-art LLM for solving undergraduate control problems. Our study serves as an initial step towards the broader goal of employing artificial general intelligence in control engineering.

Key facts. Cited by 37 papers in the map (data through 14 Oct 2026). Builds on 41 of its 60 references that are in the map. First posted on arXiv 4 Apr 2024.

Builds on

41 of 60 references.

References · 41

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2023Visual Instruction Tuningcited by 7,869
  4. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  6. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  7. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  8. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  9. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  10. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  11. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,407
  12. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,076
29 more
  1. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 886
  2. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 885
  3. 2023Reasoning with Language Model is Planning with World Modelcited by 861
  4. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 817
  5. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 783
  6. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 750
  7. 2022Large Language Models Can Self-Improvecited by 664
  8. 2022Towards Reasoning in Large Language Models: A Surveycited by 596
  9. 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 586
  10. 2022LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Modelscited by 462
  11. 2023Llemma: An Open Language Model For Mathematicscited by 388
  12. 2022A Systematic Evaluation of Large Language Models of Codecited by 363
  13. 2023On the Planning Abilities of Large Language Models : A Critical Investigationcited by 312
  14. 2023Mathematical Capabilities of ChatGPTcited by 268
  15. 2023SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Modelscited by 256
  16. 2023ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddingscited by 240
  17. 2023MathPrompter: Mathematical Reasoning using Large Language Modelscited by 236
  18. 2023Large Language Models as Commonsense Knowledge for Large-Scale Task Planningcited by 231
  19. 2023Using an LLM to Help With Code Understandingcited by 228
  20. 2024COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllabilitycited by 175
  21. 2022Ask Me Anything: A simple strategy for prompting language modelscited by 155
  22. 2023Solving Math Word Problems by Combining Language Models With Symbolic Solverscited by 124
  23. 2024GLoRe: When, Where, and How to Improve LLM Reasoning via Global and Local Refinementscited by 103
  24. 2024The Butterfly Effect of Altering Prompts: How Small Changes and Jailbreaks Affect Large Language Model Performancecited by 100
  25. 2024Coercing LLMs to do and reveal (almost) anythingcited by 89
  26. 2023LLM-Assisted Content Analysis: Using Large Language Models to Support Deductive Codingcited by 64
  27. 2023Dynamic Planning with a LLMcited by 61
  28. 2023What's the Magic Word? A Control Theory of LLM Promptingcited by 41
  29. 2023Structured Chemistry Reasoning with Large Language Modelscited by 28

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 15; 2025: 15; 2026: 7.

37 papers.

Sort:

By area:

  1. 2025Recent Advances in Large Langauge Model Benchmarks against Data Contamination: From Static to Dynamic Evaluationcited by 38
  2. 2024ControlAgent: Automating Control System Design via Novel Integration of LLM Agents and Domain Expertisecited by 21
  3. 2024MidiCaps: A large-scale MIDI dataset with text captionscited by 19
  4. 2026TSRBench: A Comprehensive Multi-task Multi-modal Time Series Reasoning Benchmark for Generalist Modelscited by 18
  5. 2024Empirical Guidelines for Deploying LLMs onto Resource-constrained Edge Devicescited by 17
  6. 2026Frontier-Eng: Benchmarking Self-Evolving Agents on Real-World Engineering Tasks with Generative Optimizationcited by 16
  7. 2024Benchmarking the Capabilities of Large Language Models in Transportation System Engineering: Accuracy, Consistency, and Reasoning Behaviorscited by 13
  8. 2025POT: Inducing Overthinking in LLMs via Black-Box Iterative Optimizationcited by 11
  9. 2024Change Is the Only Constant: Dynamic LLM Slicing based on Layer Redundancycited by 9
  10. 2025An Empirical Study of Group Conformity in Multi-Agent Systemscited by 7
  11. 2024EnviroExam: Benchmarking Environmental Science Knowledge of Large Language Modelscited by 5
  12. 2024Securing Multi-turn Conversational Language Models From Distributed Backdoor Triggerscited by 5
25 more
  1. 2024SceneLLM: Implicit Language Reasoning in LLM for Dynamic Scene Graph Generationcited by 5
  2. 2025LLM-Agent-Controller: A Universal Multi-Agent Large Language Model System as a Control Engineercited by 5
  3. 2025RLHF Fine-Tuning of LLMs for Alignment with Implicit User Feedback in Conversational Recommenderscited by 5
  4. 2024Exploring Vision Language Models for Facial Attribute Recognition: Emotion, Race, Gender, and Agecited by 4
  5. 2025MAPS: Multi-Agent Personality Shaping for Collaborative Reasoningcited by 4
  6. 2025PADBen: A Comprehensive Benchmark for Evaluating AI Text Detectors Against Paraphrase Attackscited by 4
  7. 2024Topic Classification of Case Law Using a Large Language Model and a New Taxonomy for UK Law: AI Insights into Summary Judgmentcited by 3
  8. 2024SmartLLMSentry: A Comprehensive LLM Based Smart Contract Vulnerability Detection Frameworkcited by 3
  9. 2025A Multimodal, Multilingual, and Multidimensional Pipeline for Fine-grained Crowdsourcing Earthquake Damage Evaluationcited by 3
  10. 2024Visual-Linguistic Agent: Towards Collaborative Contextual Object Reasoningcited by 2
  11. 2025The Lazy Student's Dream: ChatGPT Passing an Engineering Course on Its Owncited by 2
  12. 2025Symbolic Analysis of Grover Search Algorithm via Chain-of-Thought Reasoning and Quantum-Native Tokenizationcited by 2
  13. 2024Difficulty Estimation and Simplification of French Text Using LLMscited by 1
  14. 2025Can AI Master Construction Management (CM)? Benchmarking State-of-the-Art Large Language Models on CM Certification Examscited by 1
  15. 2024Ensuring Ground Truth Accuracy in Healthcare with the EVINCE frameworkcited by 0
  16. 2024Unveiling LLM Mechanisms Through Neural ODEs and Control Theorycited by 0
  17. 2025TMIQ: Quantifying Test and Measurement Domain Intelligence in Large Language Modelscited by 0
  18. 2025LLMs-guided adaptive compensator: Bringing Adaptivity to Automatic Control Systems with Large Language Modelscited by 0
  19. 2025Deep Learning-based Prediction of Clinical Trial Enrollment with Uncertainty Estimatescited by 0
  20. 2025TimeSeries2Report prompting enables adaptive large language model management of lithium-ion batteriescited by 0
  21. 2026Reasoning4Sciences: Bridging Reasoning Language Models to All Scientific Branchescited by 0
  22. 2026Benchmarking and Reasoning Distillation of Large Language Models for Feedback Controller Design in Complex Dynamical Systemscited by 0
  23. 2026Large Language Models in the Loop: A Stability- and Network-Aware Survey in Networked Control, Cyber-Physical, and Multi-Agent Systemscited by 0
  24. 2026Requirement-Bound Verified Commissioning: A Frozen Four-Billion-Parameter Local Model as a Candidate Generator under an External Acceptance Layer with Verification and Release Authoritycited by 0
  25. 2026SimuVerity: Benchmarking Agents for Engineering-Grade Simulink Model Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Have LLMs Advanced Enough? A Challenging Problem Solving Benchmark For Large Language Models
  2. 2025EngiBench: A Benchmark for Evaluating Large Language Models on Engineering Problem Solving
  3. 2026Mathematical Reasoning in Large Language Models: Benchmarks, Architectures, Evaluation, and Open Challenges
  4. 2024LLMs for Mathematical Modeling: Towards Bridging the Gap between Natural and Mathematical Languages
  5. 2025A Survey on Mathematical Reasoning and Optimization with Large Language Models
  6. 2025Teaching LLMs to Think Mathematically: A Critical Study of Decision-Making via Optimization
  7. 2023Competition-Level Problems are Effective LLM Evaluators
  8. 2026General365: Benchmarking General Reasoning in Large Language Models Across Diverse and Challenging Tasks