mlsanityData to 14 Oct 2026

A Multi-AI Agent System for Autonomous Optimization of Agentic AI Solutions via Iterative Refinement and LLM-Driven Feedback Loops

arXiv 2412.17149 ↗First posted 22 Dec 2024cs.CL cs.AI cs.ETcited by 13

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Agentic AI systems use specialized agents to handle tasks within complex workflows, enabling automation and efficiency. However, optimizing these systems often requires labor-intensive, manual adjustments to refine roles, tasks, and interactions. This paper introduces a framework for autonomously optimizing Agentic AI solutions across industries, such as NLP-driven enterprise applications. The system employs agents for Refinement, Execution, Evaluation, Modification, and Documentation, leveraging iterative feedback loops powered by an LLM (Llama 3.2-3B). The framework achieves optimal performance without human input by autonomously generating and testing hypotheses to improve system configurations. This approach enhances scalability and adaptability, offering a robust solution for real-world applications in dynamic environments. Case studies across diverse domains illustrate the transformative impact of this framework, showcasing significant improvements in output quality, relevance, and actionability. All data for these case studies, including original and evolved agent codes, along with their outputs, are here: https://anonymous.4open.science/r/evolver-1D11/

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 6 of its 15 references that are in the map. First posted on arXiv 22 Dec 2024.

Builds on

6 of 15 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024The Landscape of Emerging AI Agent Architectures for Reasoning, Planning, and Tool Calling: A Survey17 Apr

Other references · 5

  1. 2023MLAgentBench: Evaluating Language Agents on Machine Learning Experimentationcited by 366
  2. 2024Autonomous Evaluation and Refinement of Digital Agentscited by 137
  3. 2024AgentInstruct: Toward Generative Teaching with Agentic Flowscited by 89
  4. 2024Feedback Loops With Language Models Drive In-Context Reward Hackingcited by 77
  5. 2024Towards Agentic AI on Particle Acceleratorscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 6.

13 papers.

Sort:

By area:

  1. 2025A Survey on the Optimization of Large Language Model-based Agentscited by 46
  2. 2025An LLM-LVLM Driven Agent for Iterative and Fine-Grained Image Editingcited by 7
  3. 2026Agentic Trading: When LLM Agents Meet Financial Marketscited by 7
  4. 2026AgenticGEO: A Self-Evolving Agentic System for Generative Engine Optimizationcited by 6
  5. 2025Agentic AI for autonomous anomaly management in complex systemscited by 4
  6. 2025EvoLattice: Persistent Internal-Population Evolution through Multi-Alternative Quality-Diversity Graph Representations for LLM-Guided Program Discoverycited by 3
  7. 2026A Guide to Large Language Models in Modeling and Simulation: From Core Techniques to Critical Challengescited by 3
  8. 2026CODE-GEN: A Human-in-the-Loop RAG-Based Agentic AI System for Multiple-Choice Question Generationcited by 1
  9. 2026Anchoring Bias in LLM-as-a-Judge Systems: Prior Scores Compromise Evaluation Independencecited by 1
  10. 2025Polymorphic Combinatorial Frameworks (PCF): Guiding the Design of Mathematically-Grounded, Adaptive AI Agentscited by 0
  11. 2025Is the `Agent' Paradigm a Limiting Framework for Next-Generation Intelligent Systems?cited by 0
  12. 2025Knowledge-Guided Multi-Agent Framework for Application-Level Software Code Generationcited by 0
1 more
  1. 2026Segment-Level Agentic Topic Modeling for Improved Data Exploration and Resource Efficiencycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Evolving Excellence: Automated Optimization of LLM-based Agents
  2. 2026From Prompt-Response to Goal-Directed Systems: The Evolution of Agentic AI Software Architecture
  3. 2026Learning to Construct Practical Agentic Systems
  4. 2026Agent Gym: A Framework for Continuous Evaluation and Evolution of LLM Agents Through Human-in-the-Loop Feedback
  5. 2026AgentFactory: Towards Automated Agentic System Design and Optimization
  6. 2026OptiMAS: Automatically Optimize Multi-Agent System
  7. 2026An Empirical Study of Multi-Agent Collaboration for Automated Research
  8. 2026Co-evolving Agent Architectures and Interpretable Reasoning for Automated Optimization