# Comparative Analysis of AI Agent Architectures for Entity Relationship Classification

arXiv: https://arxiv.org/abs/2506.02426 · HTML: https://mlsanity.com/paper/2506.02426/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 17 of its 29 references that are in the map.
- First posted on arXiv 3 Jun 2025.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 17 of 17)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://mlsanity.com/paper/2110.08207/) (2021)
- [CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society](https://mlsanity.com/paper/2303.17760/) (2023)
- [Augmented Language Models: a Survey](https://mlsanity.com/paper/2302.07842/) (2023)
- [End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures](https://mlsanity.com/paper/1601.00770/) (2016)
- [Enriching Pre-trained Language Model with Entity Information for Relation Classification](https://mlsanity.com/paper/1905.08284/) (2019)
- [Injecting Domain-Specific Knowledge into Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2502.10708/) (2025)
- [Large Language Models as Financial Data Annotators: A Study on Effectiveness and Efficiency](https://mlsanity.com/paper/2403.18152/) (2024)
- [REFinD: Relation Extraction Financial Dataset](https://mlsanity.com/paper/2305.18322/) (2023)
- [Relation Extraction with Fine-Tuned Large Language Models in Retrieval Augmented Generation Frameworks](https://mlsanity.com/paper/2406.14745/) (2024)
- [AgentRE: An Agent-Based Framework for Navigating Complex Information Landscapes in Relation Extraction](https://mlsanity.com/paper/2409.01854/) (2024)
- [GPT-FinRE: In-context Learning for Financial Relation Extraction using Large Language Models](https://mlsanity.com/paper/2306.17519/) (2023)
- [Bridging Generative and Discriminative Learning: Few-Shot Relation Extraction via Two-Stage Knowledge-Guided Pre-training](https://mlsanity.com/paper/2505.12236/) (2025)
- [CORE: A Few-Shot Company Relation Classification Dataset for Robust Domain Adaptation](https://mlsanity.com/paper/2310.12024/) (2023)

## Built on by (most cited first, 1 of 1)

- [LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directions](https://mlsanity.com/paper/2509.18970/) (2025)

## Abstract (arXiv)

Entity relationship classification remains a challenging task in information extraction, especially in scenarios with limited labeled data and complex relational structures. In this study, we conduct a comparative analysis of three distinct AI agent architectures designed to perform relation classification using large language models (LLMs). The agentic architectures explored include (1) reflective self-evaluation, (2) hierarchical task decomposition, and (3) a novel multi-agent dynamic example generation mechanism, each leveraging different modes of reasoning and prompt adaptation. In particular, our dynamic example generation approach introduces real-time cooperative and adversarial prompting. We systematically compare their performance across multiple domains and model backends. Our experiments demonstrate that multi-agent coordination consistently outperforms standard few-shot prompting and approaches the performance of fine-tuned models. These findings offer practical guidance for the design of modular, generalizable LLM-based systems for structured relation extraction. The source codes and dataset are available at https://github.com/maryambrj/ALIEN.git.