# Debate as Optimization: Adaptive Conformal Prediction and Diverse Retrieval for Event Extraction

arXiv: https://arxiv.org/abs/2406.12197 · HTML: https://mlsanity.com/paper/2406.12197/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use.
- Builds on 26 of its 61 references that are in the map.
- First posted on arXiv 18 Jun 2024.

## Placed in

- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 26 of 26)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://mlsanity.com/paper/2308.08155/) (2023)
- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://mlsanity.com/paper/2305.14325/) (2023)
- [PaLM 2 Technical Report](https://mlsanity.com/paper/2305.10403/) (2023)
- [Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate](https://mlsanity.com/paper/2305.19118/) (2023)
- [ChatDev: Communicative Agents for Software Development](https://mlsanity.com/paper/2307.07924/) (2023)
- [ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate](https://mlsanity.com/paper/2308.07201/) (2023)
- [AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors](https://mlsanity.com/paper/2308.10848/) (2023)
- [Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Control](https://mlsanity.com/paper/2110.01052/) (2021)
- [ChatIE: Zero-Shot Information Extraction via Chatting with ChatGPT](https://mlsanity.com/paper/2302.10205/) (2023)
- [Improving Language Model Negotiation with Self-Play and In-Context Learning from AI Feedback](https://mlsanity.com/paper/2305.10142/) (2023)
- [InstructUIE: Multi-task Instruction Tuning for Unified Information Extraction](https://mlsanity.com/paper/2304.08085/) (2023)
- [Evaluating ChatGPT's Information Extraction Capabilities: An Assessment of Performance, Explainability, Calibration, and Faithfulness](https://mlsanity.com/paper/2304.11633/) (2023)
- [Agents: An Open-source Framework for Autonomous Language Agents](https://mlsanity.com/paper/2309.07870/) (2023)
- [Exploring the Feasibility of ChatGPT for Event Extraction](https://mlsanity.com/paper/2303.03836/) (2023)
- [Repoformer: Selective Retrieval for Repository-Level Code Completion](https://mlsanity.com/paper/2403.10059/) (2024)
- [Selection and Aggregation of Conformal Prediction Sets](https://mlsanity.com/paper/2104.13871/) (2021)
- [An Empirical Study on Information Extraction using Large Language Models](https://mlsanity.com/paper/2305.14450/) (2023)
- [Retrieval-Augmented Code Generation for Universal Information Extraction](https://mlsanity.com/paper/2311.02962/) (2023)
- [C-RAG: Certified Generation Risks for Retrieval-Augmented Language Models](https://mlsanity.com/paper/2402.03181/) (2024)
- [Learning to Break: Knowledge-Enhanced Reasoning in Multi-Agent Debate System](https://mlsanity.com/paper/2312.04854/) (2023)

## Built on by (most cited first, 10 of 10)

- [Multi-Agent Debate Strategies to Enhance Requirements Engineering with Large Language Models](https://mlsanity.com/paper/2507.05981/) (2025)
- [From Debate to Decision: Conformal Social Choice for Safe Multi-Agent Deliberation](https://mlsanity.com/paper/2604.07667/) (2026)
- [Event Extraction in Large Language Model](https://mlsanity.com/paper/2512.19537/) (2025)
- [Multi-Agent Debate Strategies: Survey, Taxonomy, and Challenges](https://mlsanity.com/paper/2607.26212/) (2026)
- [PD3: A Project Duplication Detection Framework via Adapted Multi-Agent Debate](https://mlsanity.com/paper/2505.17492/) (2025)
- [Extracting Events Like Code: A Multi-Agent Programming Framework for Zero-Shot Event Extraction](https://mlsanity.com/paper/2511.13118/) (2025)
- [Learning to Generate and Extract: A Multi-Agent Collaboration Framework For Zero-shot Document-level Event Arguments Extraction](https://mlsanity.com/paper/2603.02909/) (2026)
- [A Multi-Agent LLM Framework for Multi-Domain Low-Resource In-Context NER via Knowledge Retrieval, Disambiguation and Reflective Analysis](https://mlsanity.com/paper/2511.19083/) (2025)
- [Budgeted Act-or-Defer Multi-Agent LLM Deliberation with Local Reliability Bounds](https://mlsanity.com/paper/2606.29654/) (2026)
- [Reflective Agreement: Combining Self-Mixture of Agents with a Sequence Tagger for Robust Event Extraction](https://mlsanity.com/paper/2508.19359/) (2025)

## Abstract (arXiv)

We propose a multi-agent debate as optimization (DAO) system for event extraction, where the primary objective is to iteratively refine the large language models (LLMs) outputs through debating without parameter tuning. In DAO, we introduce two novel modules: the Diverse-RAG (DRAG) module and the Adaptive Conformal Prediction (AdaCP) module. DRAG systematically retrieves supporting information that best fits the debate discussion, while AdaCP enhances the accuracy and reliability of event extraction by effectively rejecting less promising answers. Experimental results demonstrate a significant reduction in the performance gap between supervised approaches and tuning-free LLM-based methods by 18.1% and 17.8% on ACE05 and 17.9% and 15.2% on CASIE for event detection and argument extraction respectively.