# Agent-based Automated Claim Matching with Instruction-following LLMs

arXiv: https://arxiv.org/abs/2510.23924 · HTML: https://mlsanity.com/paper/2510.23924/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use.
- Builds on 23 of its 37 references that are in the map.
- First posted on arXiv 27 Oct 2025.

## Placed in

- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 23 of 23)

- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://mlsanity.com/paper/2102.09690/) (2021)
- [Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity](https://mlsanity.com/paper/2104.08786/) (2021)
- [Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate](https://mlsanity.com/paper/2305.19118/) (2023)
- [ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate](https://mlsanity.com/paper/2308.07201/) (2023)
- [Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm](https://mlsanity.com/paper/2102.07350/) (2021)
- [A Survey on Automated Fact-Checking](https://mlsanity.com/paper/2108.11896/) (2021)
- [True Few-Shot Learning with Language Models](https://mlsanity.com/paper/2105.11447/) (2021)
- [That is a Known Lie: Detecting Previously Fact-Checked Claims](https://mlsanity.com/paper/2005.06058/) (2020)
- [A Review of Prominent Paradigms for LLM-Based Agents: Tool Use (Including RAG), Planning, and Feedback Learning](https://mlsanity.com/paper/2406.05804/) (2024)
- [Automated Fact-Checking: A Survey](https://mlsanity.com/paper/2109.11427/) (2021)
- [Overview of CheckThat! 2020: Automatic Identification and Verification of Claims in Social Media](https://mlsanity.com/paper/2007.07997/) (2020)
- [Claim Matching Beyond English to Scale Global Fact-Checking](https://mlsanity.com/paper/2106.00853/) (2021)
- [Claim Detection for Automated Fact-checking: A Survey on Monolingual, Multilingual and Cross-Lingual Research](https://mlsanity.com/paper/2401.11969/) (2024)
- [Multilingual Previously Fact-Checked Claim Retrieval](https://mlsanity.com/paper/2305.07991/) (2023)
- [FACT-GPT: Fact-Checking Augmentation via Claim Matching with LLMs](https://mlsanity.com/paper/2402.05904/) (2024)
- [Automated Claim Matching with Large Language Models: Empowering Fact-Checkers in the Fight Against Misinformation](https://mlsanity.com/paper/2310.09223/) (2023)
- [The Role of Context in Detecting Previously Fact-Checked Claims](https://mlsanity.com/paper/2104.07423/) (2021)
- [Matching Tweets With Applicable Fact-Checks Across Languages](https://mlsanity.com/paper/2202.07094/) (2022)
- [Just rephrase it! Uncertainty estimation in closed-source language models via multiple rephrased queries](https://mlsanity.com/paper/2405.13907/) (2024)
- [Zero-shot and Few-shot Learning with Instruction-following LLMs for Claim Matching in Automated Fact-checking](https://mlsanity.com/paper/2501.10860/) (2025)

## Abstract (arXiv)

We present a novel agent-based approach for the automated claim matching task with instruction-following LLMs. We propose a two-step pipeline that first generates prompts with LLMs, to then perform claim matching as a binary classification task with LLMs. We demonstrate that LLM-generated prompts can outperform SOTA with human-generated prompts, and that smaller LLMs can do as well as larger ones in the generation process, allowing to save computational resources. We also demonstrate the effectiveness of using different LLMs for each step of the pipeline, i.e. using an LLM for prompt generation, and another for claim matching. Our investigation into the prompt generation process in turn reveals insights into the LLMs' understanding of claim matching.