mlsanityData to 7 Oct 2026

Middleware for LLMs: Tools Are Instrumental for Language Agents in Complex Environments

arXiv 2402.14672 ↗First posted 22 Feb 2024cs.CL cs.AIcited by 39

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

The applications of large language models (LLMs) have expanded well beyond the confines of text processing, signaling a new era where LLMs are envisioned as generalist agents capable of operating within complex environments. These environments are often highly expansive, making it impossible for the LLM to process them within its short-term memory. Motivated by recent research on extending the capabilities of LLMs with tools, we seek to investigate the intriguing potential of tools to augment LLMs in handling such complexity by introducing a novel class of tools, termed middleware, to aid in the proactive exploration within these massive environments. Such specialized tools can serve as a middleware layer shielding the LLM from environmental complexity. In two representative complex environments -- knowledge bases (KBs) and databases -- we demonstrate the significant potential of augmenting language agents with tools in complex environments. Notably, equipped with the middleware, GPT-4 achieves 2.8X the performance of the best baseline in tasks requiring access to database content and 2.2X in KB tasks. Our findings illuminate the path for advancing language agents in real-world applications.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 33 of its 47 references that are in the map. First posted on arXiv 22 Feb 2024.

Builds on

33 of 47 references.

References · 33

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  6. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  7. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,210
  8. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  9. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  10. 2023Teaching Large Language Models to Self-Debugcited by 963
  11. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 751
  12. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 679
21 more
  1. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
  2. 2023DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correctioncited by 465
  3. 2018The Web as a Knowledge-base for Answering Complex Questionscited by 409
  4. 2023Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluationcited by 378
  5. 2023StructGPT: A General Framework for Large Language Model to Reason over Structured Datacited by 339
  6. 2023Tool Learning with Foundation Modelscited by 302
  7. 2023ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddingscited by 238
  8. 2023Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planningcited by 204
  9. 2017Variational Reasoning for Question Answering with Knowledge Graphcited by 170
  10. 2020Beyond I.I.D.: Three Levels of Generalization for Question Answering on Knowledge Basescited by 169
  11. 2019A Comprehensive Exploration on WikiSQL with Table-Aware Word Contextualizationcited by 88
  12. 2022DecAF: Joint Decoding of Answers and Logical Forms for Question Answering over Knowledge Basescited by 83
  13. 2020KQA Pro: A Dataset with Explicit Compositional Programs for Complex Question Answering over Knowledge Basecited by 76
  14. 2023Few-shot In-context Learning for Knowledge Base Question Answeringcited by 74
  15. 2024TroVE: Inducing Verifiable and Efficient Toolboxes for Solving Programmatic Taskscited by 70
  16. 2023Exploring Chain-of-Thought Style Prompting for Text-to-SQLcited by 69
  17. 2022Don't Generate, Discriminate: A Proposal for Grounding Language Models to Real-World Environmentscited by 57
  18. 2022ArcaneQA: Dynamic Program Induction and Contextualized Encoding for Knowledge Base Question Answeringcited by 48
  19. 2021Grounding 'Grounding' in NLPcited by 44
  20. 2022Knowledge Base Question Answering: A Semantic Parsing Perspectivecited by 22
  21. 2023Battle of the Large Language Models: Dolly vs LLaMA vs Vicuna vs Guanaco vs Bard vs ChatGPT -- A Text-to-SQL Parsing Comparisoncited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 18; 2025: 16; 2026: 5.

39 papers.

Sort:

By area:

  1. 2024WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learningcited by 243
  2. 2024Tool Learning with Large Language Models: A Surveycited by 221
  3. 2025SkillWeaver: Web Agents can Self-Improve by Discovering and Honing Skillscited by 174
  4. 2024Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQLcited by 118
  5. 2025Towards Scientific Intelligence: A Survey of LLM-based Scientific Agentscited by 96
  6. 2025AGrail: A Lifelong Agent Guardrail with Effective and Adaptive Safety Detectioncited by 82
  7. 2025Agent Learning via Early Experiencecited by 79
  8. 2024Beyond Browsing: API-Based Web Agentscited by 77
  9. 2024A Survey of Large Language Models for Graphscited by 71
  10. 2024HyperAgent: Generalist Software Engineering Agents to Solve Coding Tasks at Scalecited by 61
  11. 2025Evaluating LLM-based Agents for Multi-Turn Conversations: A Surveycited by 56
  12. 2024Before Generation, Align it! A Novel and Effective Strategy for Mitigating Hallucinations in Text-to-SQL Generationcited by 49
27 more
  1. 2025A Survey on the Optimization of Large Language Model-based Agentscited by 45
  2. 2024Interactive-KBQA: Multi-Turn Interactions for Knowledge Base Question Answering with Large Language Modelscited by 25
  3. 2024Revealing the Barriers of Language Agents in Planningcited by 20
  4. 2026REMem: Reasoning with Episodic Memory in Language Agentcited by 20
  5. 2024LLMs in the Imaginarium: Tool Learning through Simulated Trial and Errorcited by 19
  6. 2024Tapilot-Crossing: Benchmarking and Evolving LLMs Towards Interactive Data Analysis Agentscited by 17
  7. 2025MCPVerse: An Expansive, Real-World Benchmark for Agentic Tool Usecited by 16
  8. 2024From Natural Language to SQL: Review of LLM-based Text-to-SQL Systemscited by 15
  9. 2025SHARE: An SLM-based Hierarchical Action CorREction Assistant for Text-to-SQLcited by 15
  10. 2025AskToAct: Enhancing LLMs Tool Use via Self-Correcting Clarificationcited by 12
  11. 2024Can Tool-augmented Large Language Models be Aware of Incomplete Conditions?cited by 8
  12. 2024A Perspective for Adapting Generalist AI to Specialized Medical AI Applications and Their Challengescited by 8
  13. 2025Reason-Align-Respond: Aligning LLM Reasoning with Knowledge Graphs for KGQAcited by 7
  14. 2025Efficient numeracy in language models through single-token number embeddingscited by 6
  15. 2024TARGA: Targeted Synthetic Data Generation for Practical Reasoning over Structured Datacited by 5
  16. 2025ALRPHFS: Adversarially Learned Risk Patterns with Hierarchical Fast & Slow Reasoning for Robust Agent Defensecited by 5
  17. 2025GraphChain: Large Language Models for Large-scale Graph Analysis via Tool Chainingcited by 5
  18. 2024A Taxonomy of Architecture Options for Foundation Model-based Agents: Analysis and Decision Modelcited by 4
  19. 2024Tool-Assisted Agent on SQL Inspection and Refinement in Real-World Scenarioscited by 4
  20. 2025Exploring the Landscape of Text-to-SQL with Large Language Models: Progresses, Challenges and Opportunitiescited by 4
  21. 2025CLEAR-KGQA: Clarification-Enhanced Ambiguity Resolution for Knowledge Graph Question Answeringcited by 3
  22. 2024Multi-Turn Interactions for Text-to-SQL with Large Language Modelscited by 2
  23. 2025Micro-Act: Mitigating Knowledge Conflict in LLM-based RAG via Actionable Self-Reasoningcited by 1
  24. 2026Push Your Agent: Measuring and Enforcing Quantitative Goal Persistence in Long-Horizon LLM Agentscited by 1
  25. 2026SearchOS-V1: Towards Robust Open-Domain Information-Seeking Agent Collaborationcited by 1
  26. 2026Execution-First Synthetic Tool-Use Trace Generation for LLM Agentscited by 1
  27. 2026Machine Learning-Based Detection of MCP Attackscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Small LLMs Are Weak Tool Learners: A Multi-LLM Agent
  2. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challenges
  3. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challenges
  4. 2024A Review of Prominent Paradigms for LLM-Based Agents: Tool Use (Including RAG), Planning, and Feedback Learning
  5. 2024Creating an LLM-based AI-agent: A high-level methodology towards enhancing LLMs with APIs
  6. 2024Large Language Model-Based Agents for Software Engineering: A Survey
  7. 2024Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospects
  8. 2023AgentBench: Evaluating LLMs as Agents