mlsanityData to 7 Oct 2026

Eliciting Problem Specifications via Large Language Models

arXiv 2405.12147 ↗First posted 20 May 2024cs.AI cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

Cognitive systems generally require a human to translate a problem definition into some specification that the cognitive system can use to attempt to solve the problem or perform the task. In this paper, we illustrate that large language models (LLMs) can be utilized to map a problem class, defined in natural language, into a semi-formal specification that can then be utilized by an existing reasoning and learning system to solve instances from the problem class. We present the design of LLM-enabled cognitive task analyst agent(s). Implemented with LLM agents, this system produces a definition of problem spaces for tasks specified in natural language. LLM prompts are derived from the definition of problem spaces in the AI literature and general problem-solving strategies (Polya's How to Solve It). A cognitive system can then use the problem-space specification, applying domain-general problem solving strategies ("weak methods" such as search), to solve multiple instances of problems from the problem class. This result, while preliminary, suggests the potential for speeding cognitive systems research via disintermediation of problem formulation while also retaining core capabilities of cognitive systems, such as robust inference and online learning.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 14 of its 45 references that are in the map. First posted on arXiv 20 May 2024.

Builds on

14 of 45 references.

References · 14

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  4. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  5. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  6. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
  7. 2023Large Language Models for Software Engineering: A Systematic Literature Reviewcited by 389
  8. 2023Language Models can Solve Computer Taskscited by 371
  9. 2023Tool Learning with Foundation Modelscited by 302
  10. 2023ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Modelscited by 174
  11. 2023SatLM: Satisfiability-Aided Language Models Using Declarative Promptingcited by 100
  12. 2023On the Planning Abilities of Large Language Models (A Critical Investigation with a Proposed Benchmark)cited by 85
2 more
  1. 2023How to use LLMs for Text Analysiscited by 5
  2. 2023Improving Knowledge Extraction from LLMs for Task Learning through Agent Analysiscited by 1

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025NL2GenSym: Natural Language to Generative Symbolic Rules for SOAR Cognitive Architecture via Large Language Modelscited by 1
  2. 2025LLM-Augmented Symbolic NLU System for More Reliable Continuous Causal Statement Interpretationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Exploiting Language Models as a Source of Knowledge for Cognitive Agents
  2. 2026Cognitive Agent Compilation for Explicit Problem Solver Modeling
  3. 2026Hypothesis-guided discovery of cognitive algorithms via program refinement
  4. 2025From Language to Action: A Review of Large Language Models as Autonomous Agents and Tool Users
  5. 2025Distilling LLM Agent into Small Models with Retrieval and Code Tools
  6. 2026Cognitive Models and AI Algorithms Provide Templates for Designing Language Agents
  7. 2024Navigating Complexity: Orchestrated Problem Solving with Multi-Agent LLMs
  8. 2023StrategyLLM: Large Language Models as Strategy Generators, Executors, Optimizers, and Evaluators for Problem Solving