# Efficient Leave-one-out Approximation in LLM Multi-agent Debate Based on Introspection

arXiv: https://arxiv.org/abs/2505.22192 · HTML: https://mlsanity.com/paper/2505.22192/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use.
- Builds on 12 of its 19 references that are in the map.
- First posted on arXiv 28 May 2025.

## Placed in

- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 12 of 12)

- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://mlsanity.com/paper/2305.14325/) (2023)
- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023)
- [ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate](https://mlsanity.com/paper/2308.07201/) (2023)
- [LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion](https://mlsanity.com/paper/2306.02561/) (2023)
- [MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning](https://mlsanity.com/paper/2311.10537/) (2023)
- [Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting](https://mlsanity.com/paper/2306.17563/) (2023)
- [More Agents Is All You Need](https://mlsanity.com/paper/2402.05120/) (2024)
- [Examining Inter-Consistency of Large Language Models Collaboration: An In-depth Analysis via Debate](https://mlsanity.com/paper/2305.11595/) (2023)
- [Computational Experiments Meet Large Language Model Based Agents: A Survey and Perspective](https://mlsanity.com/paper/2402.00262/) (2024)
- [Prompt Valuation Based on Shapley Values](https://mlsanity.com/paper/2312.15395/) (2023)
- [A Survey on Contribution Evaluation in Vertical Federated Learning](https://mlsanity.com/paper/2405.02364/) (2024)
- [Hierarchical Federated Learning Incentivization for Gas Usage Estimation](https://mlsanity.com/paper/2307.00233/) (2023)

## Built on by (most cited first, 4 of 4)

- [Branch-and-Browse: Efficient and Controllable Web Exploration with Tree-Structured Reasoning and Action Memory](https://mlsanity.com/paper/2510.19838/) (2025)
- [Multi-Agent Debate Strategies: Survey, Taxonomy, and Challenges](https://mlsanity.com/paper/2607.26212/) (2026)
- [The impact of multi-agent debate protocols on debate quality: a controlled case study](https://mlsanity.com/paper/2603.28813/) (2026)
- [Agents that Matter: Optimizing Multi-Agent LLMs via Removal-Based Attribution](https://mlsanity.com/paper/2605.27621/) (2026)

## Abstract (arXiv)

Multi-agent systems based on large language models (LLMs) advance automatic task completion in various fields, where debate is a common cooperation form for agents to solve complicated problems with reasoning and cross-review to solidify answers. Assessing the individual contributions of agents within these debates is crucial for system refinement and outcome reliability. Traditional leave-one-out (LOO) method offers a clear framework for evaluating each agent's role but face challenges in LLM-based systems due to high computational costs and associated financial implications. This paper presents introspective-leave-one-out (IntrospecLOO), a simple yet effective prompting for approximation of LOO in LLM-powered multi-agent debates. IntrospecLOO introduces an additional querying round after standard debates, prompting agents to update their answers while ignoring responses from a designated agent. This strategy effectively isolates and gauges each participant's influence at a reduced query complexity compared to the original LOO approaches. Validation through experiments on three benchmark datasets confirms the effectiveness of IntrospecLOO.