# HerO at AVeriTeC: The Herd of Open Large Language Models for Verifying Real-World Claims

arXiv: https://arxiv.org/abs/2410.12377 · HTML: https://mlsanity.com/paper/2410.12377/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 23 references that are in the map.
- First posted on arXiv 16 Oct 2024.

## Builds on (most cited first, 14 of 14)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [LaMDA: Language Models for Dialog Applications](https://mlsanity.com/paper/2201.08239/) (2022)
- [Precise Zero-Shot Dense Retrieval without Relevance Labels](https://mlsanity.com/paper/2212.10496/) (2022)
- [A Survey on Automated Fact-Checking](https://mlsanity.com/paper/2108.11896/) (2021)
- ["Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection](https://mlsanity.com/paper/1705.00648/) (2017)
- [MultiFC: A Real-World Multi-Domain Dataset for Evidence-Based Fact Checking of Claims](https://mlsanity.com/paper/1909.03242/) (2019)
- [FEVEROUS: Fact Extraction and VERification Over Unstructured and Structured information](https://mlsanity.com/paper/2106.05707/) (2021)
- [AVeriTeC: A Dataset for Real-world Claim Verification with Evidence from the Web](https://mlsanity.com/paper/2305.13117/) (2023)
- [The Fact Extraction and VERification (FEVER) Shared Task](https://mlsanity.com/paper/1811.10971/) (2018)

## Built on by (most cited first, 12 of 12)

- [Debating Truth: Debate-driven Claim Verification with Multiple Large Language Model Agents](https://mlsanity.com/paper/2507.19090/) (2025)
- [Team HUMANE at AVeriTeC 2025: HerO 2 for Efficient Fact Verification](https://mlsanity.com/paper/2507.11004/) (2025)
- [Fact2Fiction: Targeted Poisoning Attack to Agentic Fact-checking System](https://mlsanity.com/paper/2508.06059/) (2025)
- [Attacks by Content: Automated Fact-checking is an AI Security Issue](https://mlsanity.com/paper/2510.11238/) (2025)
- [AIC CTU@FEVER 8: On-premise fact checking through long context RAG](https://mlsanity.com/paper/2508.04390/) (2025)
- [Multi-Sourced, Multi-Agent Evidence Retrieval for Fact-Checking](https://mlsanity.com/paper/2603.00267/) (2026)
- [The Automatic Verification of Image-Text Claims (AVerImaTeC) Shared Task](https://mlsanity.com/paper/2602.11221/) (2026)
- [From Verdict to Process: Agentic Reinforcement Learning for Multi-Stage Fact Verification](https://mlsanity.com/paper/2606.13262/) (2026)
- [PledgeTracker: A System for Monitoring the Fulfilment of Pledges](https://mlsanity.com/paper/2509.11804/) (2025)
- [ClaimCheck: Real-Time Fact-Checking with Small Language Models](https://mlsanity.com/paper/2510.01226/) (2025)
- [VILLAIN at AVerImaTeC: Verifying Image-Text Claims via Multi-Agent Collaboration](https://mlsanity.com/paper/2602.04587/) (2026)
- [CAAFC: Chronological Actionable Automated Fact-Checker for misinformation / non-factual hallucination detection and correction](https://mlsanity.com/paper/2605.12436/) (2026)

## Abstract (arXiv)

To tackle the AVeriTeC shared task hosted by the FEVER-24, we introduce a system that only employs publicly available large language models (LLMs) for each step of automated fact-checking, dubbed the Herd of Open LLMs for verifying real-world claims (HerO). For evidence retrieval, a language model is used to enhance a query by generating hypothetical fact-checking documents. We prompt pretrained and fine-tuned LLMs for question generation and veracity prediction by crafting prompts with retrieved in-context samples. HerO achieved 2nd place on the leaderboard with the AVeriTeC score of 0.57, suggesting the potential of open LLMs for verifying real-world claims. For future research, we make our code publicly available at https://github.com/ssu-humane/HerO.