# A Multi-AI Agent System for Autonomous Optimization of Agentic AI Solutions via Iterative Refinement and LLM-Driven Feedback Loops

arXiv: https://arxiv.org/abs/2412.17149 · HTML: https://mlsanity.com/paper/2412.17149/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 6 of its 15 references that are in the map.
- First posted on arXiv 22 Dec 2024.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 6 of 6)

- [MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation](https://mlsanity.com/paper/2310.03302/) (2023)
- [The Landscape of Emerging AI Agent Architectures for Reasoning, Planning, and Tool Calling: A Survey](https://mlsanity.com/paper/2404.11584/) (2024)
- [Autonomous Evaluation and Refinement of Digital Agents](https://mlsanity.com/paper/2404.06474/) (2024)
- [AgentInstruct: Toward Generative Teaching with Agentic Flows](https://mlsanity.com/paper/2407.03502/) (2024)
- [Feedback Loops With Language Models Drive In-Context Reward Hacking](https://mlsanity.com/paper/2402.06627/) (2024)
- [Towards Agentic AI on Particle Accelerators](https://mlsanity.com/paper/2409.06336/) (2024)

## Built on by (most cited first, 13 of 13)

- [A Survey on the Optimization of Large Language Model-based Agents](https://mlsanity.com/paper/2503.12434/) (2025)
- [An LLM-LVLM Driven Agent for Iterative and Fine-Grained Image Editing](https://mlsanity.com/paper/2508.17435/) (2025)
- [Agentic Trading: When LLM Agents Meet Financial Markets](https://mlsanity.com/paper/2605.19337/) (2026)
- [AgenticGEO: A Self-Evolving Agentic System for Generative Engine Optimization](https://mlsanity.com/paper/2603.20213/) (2026)
- [Agentic AI for autonomous anomaly management in complex systems](https://mlsanity.com/paper/2507.15676/) (2025)
- [EvoLattice: Persistent Internal-Population Evolution through Multi-Alternative Quality-Diversity Graph Representations for LLM-Guided Program Discovery](https://mlsanity.com/paper/2512.13857/) (2025)
- [A Guide to Large Language Models in Modeling and Simulation: From Core Techniques to Critical Challenges](https://mlsanity.com/paper/2602.05883/) (2026)
- [CODE-GEN: A Human-in-the-Loop RAG-Based Agentic AI System for Multiple-Choice Question Generation](https://mlsanity.com/paper/2604.03926/) (2026)
- [Anchoring Bias in LLM-as-a-Judge Systems: Prior Scores Compromise Evaluation Independence](https://mlsanity.com/paper/2608.25869/) (2026)
- [Polymorphic Combinatorial Frameworks (PCF): Guiding the Design of Mathematically-Grounded, Adaptive AI Agents](https://mlsanity.com/paper/2508.01581/) (2025)
- [Is the `Agent' Paradigm a Limiting Framework for Next-Generation Intelligent Systems?](https://mlsanity.com/paper/2509.10875/) (2025)
- [Knowledge-Guided Multi-Agent Framework for Application-Level Software Code Generation](https://mlsanity.com/paper/2510.19868/) (2025)
- [Segment-Level Agentic Topic Modeling for Improved Data Exploration and Resource Efficiency](https://mlsanity.com/paper/2609.31460/) (2026)

## Abstract (arXiv)

Agentic AI systems use specialized agents to handle tasks within complex workflows, enabling automation and efficiency. However, optimizing these systems often requires labor-intensive, manual adjustments to refine roles, tasks, and interactions. This paper introduces a framework for autonomously optimizing Agentic AI solutions across industries, such as NLP-driven enterprise applications. The system employs agents for Refinement, Execution, Evaluation, Modification, and Documentation, leveraging iterative feedback loops powered by an LLM (Llama 3.2-3B). The framework achieves optimal performance without human input by autonomously generating and testing hypotheses to improve system configurations. This approach enhances scalability and adaptability, offering a robust solution for real-world applications in dynamic environments. Case studies across diverse domains illustrate the transformative impact of this framework, showcasing significant improvements in output quality, relevance, and actionability. All data for these case studies, including original and evolved agent codes, along with their outputs, are here: https://anonymous.4open.science/r/evolver-1D11/