# To Ensemble or Not: Assessing Majority Voting Strategies for Phishing Detection with Large Language Models

arXiv: https://arxiv.org/abs/2412.00166 · HTML: https://mlsanity.com/paper/2412.00166/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Nov 2024.

## Built on by (most cited first, 12 of 12)

- [Towards Robust Agentic CUDA Kernel Benchmarking, Verification, and Optimization](https://mlsanity.com/paper/2509.14279/) (2025)
- [Large Multimodal Agents for Accurate Phishing Detection with Enhanced Token Optimization and Cost Reduction](https://mlsanity.com/paper/2412.02301/) (2024)
- [Cost-aware LLM-based Online Dataset Annotation](https://mlsanity.com/paper/2505.15101/) (2025)
- [Detecting Quishing Attacks with Machine Learning Techniques Through QR Code Analysis](https://mlsanity.com/paper/2505.03451/) (2025)
- [Belief-Calibrated Multi-Agent Consensus Seeking for Complex NLP Tasks](https://mlsanity.com/paper/2510.06307/) (2025)
- [Prompt Attack Detection with LLM-as-a-Judge and Mixture-of-Models](https://mlsanity.com/paper/2603.25176/) (2026)
- [CLASP: Cost-Optimized LLM-based Agentic System for Phishing Detection](https://mlsanity.com/paper/2510.18585/) (2025)
- [Retrieval-Augmented Few-Shot Prompting Versus Fine-Tuning for Code Vulnerability Detection](https://mlsanity.com/paper/2512.04106/) (2025)
- [When Does Synthetic CT Transfer? A Label-Free Donor/Host Diagnostic for Medical Vision-Language Model Routing on Real Lung CT](https://mlsanity.com/paper/2606.29232/) (2026)
- [Video2Reaction: Mapping Video to Audience Reaction Distribution in the Wild](https://mlsanity.com/paper/2607.06875/) (2026)
- [Evaluating Large Language Models for Symbolic Security Protocol Analysis](https://mlsanity.com/paper/2607.20712/) (2026)
- [Video2Reaction: Training Foundation Video Models to Predict Audience Reaction](https://mlsanity.com/paper/2609.01816/) (2026)

## Abstract (arXiv)

The effectiveness of Large Language Models (LLMs) significantly relies on the quality of the prompts they receive. However, even when processing identical prompts, LLMs can yield varying outcomes due to differences in their training processes. To leverage the collective intelligence of multiple LLMs and enhance their performance, this study investigates three majority voting strategies for text classification, focusing on phishing URL detection. The strategies are: (1) a prompt-based ensemble, which utilizes majority voting across the responses generated by a single LLM to various prompts; (2) a model-based ensemble, which entails aggregating responses from multiple LLMs to a single prompt; and (3) a hybrid ensemble, which combines the two methods by sending different prompts to multiple LLMs and then aggregating their responses. Our analysis shows that ensemble strategies are most suited in cases where individual components exhibit equivalent performance levels. However, when there is a significant discrepancy in individual performance, the effectiveness of the ensemble method may not exceed that of the highest-performing single LLM or prompt. In such instances, opting for ensemble techniques is not recommended.