# Retrieval- and Argumentation-Enhanced Multi-Agent LLMs for Judgmental Forecasting (Extended Version with Supplementary Material)

arXiv: https://arxiv.org/abs/2510.24303 · HTML: https://mlsanity.com/paper/2510.24303/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use.
- Builds on 21 of its 48 references that are in the map.
- First posted on arXiv 28 Oct 2025.

## Placed in

- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 21 of 21)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Gemma 2: Improving Open Language Models at a Practical Size](https://mlsanity.com/paper/2408.00118/) (2024)
- [Mixtral of Experts](https://mlsanity.com/paper/2401.04088/) (2024)
- [Gemma: Open Models Based on Gemini Research and Technology](https://mlsanity.com/paper/2403.08295/) (2024)
- [Retrieval-Augmented Generation for AI-Generated Content: A Survey](https://mlsanity.com/paper/2402.19473/) (2024)
- [Approaching Human-Level Forecasting with Language Models](https://mlsanity.com/paper/2402.18563/) (2024)
- [Towards LLM-based Fact Verification on News Claims with a Hierarchical Step-by-Step Prompting Method](https://mlsanity.com/paper/2310.00305/) (2023)
- [Evolution of Semantic Similarity -- A Survey](https://mlsanity.com/paper/2004.13820/) (2020)
- [Explainable Claim Verification via Knowledge-Grounded Reasoning with Large Language Models](https://mlsanity.com/paper/2310.05253/) (2023)
- [Argumentative Large Language Models for Explainable and Contestable Claim Verification](https://mlsanity.com/paper/2405.02079/) (2024)
- [Retrieval Augmented Fact Verification by Synthesizing Contrastive Arguments](https://mlsanity.com/paper/2406.09815/) (2024)
- [Interactive Explanations by Conflict Resolution via Argumentative Exchanges](https://mlsanity.com/paper/2303.15022/) (2023)
- [Large Language Model Prediction Capabilities: Evidence from a Real-World Forecasting Tournament](https://mlsanity.com/paper/2310.13014/) (2023)
- [Claim Verification in the Age of Large Language Models: A Survey](https://mlsanity.com/paper/2408.14317/) (2024)
- [Can Large Language Models perform Relation-based Argument Mining?](https://mlsanity.com/paper/2402.11243/) (2024)
- [AI-Augmented Predictions: LLM Assistants Improve Human Forecasting Accuracy](https://mlsanity.com/paper/2402.07862/) (2024)
- [Resolving Conflicting Evidence in Automated Fact-Checking: A Study on Retrieval-Augmented LLMs](https://mlsanity.com/paper/2505.17762/) (2025)
- [ArgRAG: Explainable Retrieval Augmented Generation using Quantitative Bipolar Argumentation](https://mlsanity.com/paper/2508.20131/) (2025)
- [Humans vs Large Language Models: Judgmental Forecasting in an Era of Advanced AI](https://mlsanity.com/paper/2312.06941/) (2023)
- [Argumentatively Coherent Judgmental Forecasting](https://mlsanity.com/paper/2507.23163/) (2025)
- [CRAVE: A Conflicting Reasoning Approach for Explainable Claim Verification Using LLMs](https://mlsanity.com/paper/2504.14905/) (2025)

## Built on by (most cited first, 8 of 8)

- [Argumentation for Explainable and Globally Contestable Decision Support with LLMs](https://mlsanity.com/paper/2603.14643/) (2026)
- [Multi-Agent Dialectical Refinement for Enhanced Argument Classification](https://mlsanity.com/paper/2603.27451/) (2026)
- [ARGORA: Orchestrated Argumentation for Causally Grounded LLM Reasoning and Decision Making](https://mlsanity.com/paper/2601.21533/) (2026)
- [From Argument Components to Graphs: A Multi-Agent Debate with Confidence Gating for Argument Relations](https://mlsanity.com/paper/2606.16047/) (2026)
- [Towards an Argumentative Foundation for Evaluative AI](https://mlsanity.com/paper/2608.07473/) (2026)
- [LLM-based Agents for Forecasting and Prediction: Methods, Training, Evaluation, and Applications](https://mlsanity.com/paper/2608.23058/) (2026)
- [ArgLLM-App: An Interactive System for Argumentative Reasoning with Large Language Models](https://mlsanity.com/paper/2602.24172/) (2026)
- [Argumentation for Common Ground: Finding Zones of Possible Agreement between Individuals in Conflict](https://mlsanity.com/paper/2608.15634/) (2026)

## Abstract (arXiv)

Judgmental forecasting is the task of making predictions about future events based on human judgment. This task can be seen as a form of claim verification, where the claim corresponds to a future event and the task is to assess the plausibility of that event. In this paper, we propose a novel multi-agent framework for claim verification, whereby different agents may disagree on claim veracity and bring specific evidence for and against the claims, represented as quantitative bipolar argumentation frameworks (QBAFs). We then instantiate the framework for supporting claim verification, with a variety of agents realised with Large Language Models (LLMs): (1) ArgLLM agents, an existing approach for claim verification that generates and evaluates QBAFs; (2) RbAM agents, whereby LLM-empowered Relation-based Argument Mining (RbAM) from external sources is used to generate QBAFs; (3) RAG-ArgLLM agents, extending ArgLLM agents with a form of Retrieval-Augmented Generation (RAG) of arguments from external sources. Finally, we conduct experiments with two standard judgmental forecasting datasets, with instances of our framework with two or three agents, empowered by six different base LLMs. We observe that combining evidence from agents can improve forecasting accuracy, especially in the case of three agents, while providing an explainable combination of evidence for claim verification.