mlsanityData to 14 Oct 2026

Detecting Voice Phishing with Precision: Fine-Tuning Small Language Models

arXiv 2506.06180 ↗First posted 6 Jun 2025cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

We develop a voice phishing (VP) detector by fine-tuning Llama3, a representative open-source, small language model (LM). In the prompt, we provide carefully-designed VP evaluation criteria and apply the Chain-of-Thought (CoT) technique. To evaluate the robustness of LMs and highlight differences in their performance, we construct an adversarial test dataset that places the models under challenging conditions. Moreover, to address the lack of VP transcripts, we create transcripts by referencing existing or new types of VP techniques. We compare cases where evaluation criteria are included, the CoT technique is applied, or both are used together. In the experiment, our results show that the Llama3-8B model, fine-tuned with a dataset that includes a prompt with VP evaluation criteria, yields the best performance among small LMs and is comparable to that of a GPT-4-based VP detector. These findings indicate that incorporating human expert knowledge into the prompt is more effective than using the CoT technique for small LMs in VP detection.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 10 of its 24 references that are in the map. First posted on arXiv 6 Jun 2025.

Builds on

10 of 24 references.

References · 10

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  5. 2022Large Language Models Are Reasoning Teacherscited by 329
  6. 2022Teaching Small Language Models to Reasoncited by 284
  7. 2024Accessing GPT-4 level Mathematical Olympiad Solutions via Monte Carlo Tree Self-refine with LLaMa-3 8Bcited by 158
  8. 2024A Comprehensive Survey of Small Language Models in the Era of Large Language Models: Techniques, Enhancements, Applications, Collaboration with LLMs, and Trustworthinesscited by 127
  9. 2024AD-LLM: Benchmarking Large Language Models for Anomaly Detectioncited by 26
  10. 2024Optimizing Language Augmentation for Multilingual Large Language Models: A Case Study on Koreancited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Anatomy of a Scam Call: What 10,000 real scam and spam calls reveal about how phone scammers operatecited by 1
  2. 2026Comparing Self-Supervised and Domain-Invariant Features for Cross-Domain Voice Phishing Detectioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Phishing Detection in the Gen-AI Era: Quantized LLMs vs Classical Models
  2. 2026Diagnosing Generalization Failures in Fine-Tuned LLMs: A Cross-Architectural Study on Phishing Detection
  3. 2025Phishing Email Detection Using Large Language Models
  4. 2026GuardPhish: Securing Open-Source LLMs from Phishing Abuse
  5. 2023From Chatbots to PhishBots? -- Preventing Phishing scams created using ChatGPT, Google Bard and Claude
  6. 2025Safeguarding Efficacy in Large Language Models: Evaluating Resistance to Human-Written and Algorithmic Adversarial Prompts
  7. 2026Clouding the Mirror: Stealthy Prompt Injection Attacks Targeting LLM-based Phishing Detection
  8. 2025Evaluating Large Language Models for Phishing Detection, Self-Consistency, Faithfulness, and Explainability