# Eliciting Least-to-Most Reasoning for Phishing URL Detection

arXiv: https://arxiv.org/abs/2601.20270 · HTML: https://mlsanity.com/paper/2601.20270/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 12 of its 31 references that are in the map.
- First posted on arXiv 28 Jan 2026.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)

## Builds on (most cited first, 12 of 12)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [Least-to-Most Prompting Enables Complex Reasoning in Large Language Models](https://mlsanity.com/paper/2205.10625/) (2022)
- [Gemma: Open Models Based on Gemini Research and Technology](https://mlsanity.com/paper/2403.08295/) (2024)
- [URLNet: Learning a URL Representation with Deep Learning for Malicious URL Detection](https://mlsanity.com/paper/1802.03162/) (2018)
- [URLTran: Improving Phishing URL Detection Using Transformers](https://mlsanity.com/paper/2106.05256/) (2021)
- [Detecting Phishing Sites Using ChatGPT](https://mlsanity.com/paper/2306.05816/) (2023)
- [Multimodal Large Language Models for Phishing Webpage Detection and Identification](https://mlsanity.com/paper/2408.05941/) (2024)
- [LLMs are One-Shot URL Classifiers and Explainers](https://mlsanity.com/paper/2409.14306/) (2024)
- [Web Page Classification using LLMs for Crawling Support](https://mlsanity.com/paper/2505.06972/) (2025)

## Abstract (arXiv)

Phishing continues to be one of the most prevalent attack vectors, making accurate classification of phishing URLs essential. Recently, large language models (LLMs) have demonstrated promising results in phishing URL detection. However, their reasoning capabilities that enabled such performance remain underexplored. To this end, in this paper, we propose a Least-to-Most prompting framework for phishing URL detection. In particular, we introduce an "answer sensitivity" mechanism that guides Least-to-Most's iterative approach to enhance reasoning and yield higher prediction accuracy. We evaluate our framework using three URL datasets and four state-of-the-art LLMs, comparing against a one-shot approach and a supervised model. We demonstrate that our framework outperforms the one-shot baseline while achieving performance comparable to that of the supervised model, despite requiring significantly less training data. Furthermore, our in-depth analysis highlights how the iterative reasoning enabled by Least-to-Most, and reinforced by our answer sensitivity mechanism, drives these performance gains. Overall, we show that this simple yet powerful prompting strategy consistently outperforms both one-shot and supervised approaches, despite requiring minimal training or few-shot guidance. Our experimental setup can be found in our Github repository github.sydney.edu.au/htri0928/least-to-most-phishing-detection.