# Performance Review on LLM for solving leetcode problems

arXiv: https://arxiv.org/abs/2502.15770 · HTML: https://mlsanity.com/paper/2502.15770/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 17 references that are in the map.
- First posted on arXiv 16 Feb 2025.

## Builds on (most cited first, 6 of 6)

- [A Performance Study of LLM-Generated Code on Leetcode](https://mlsanity.com/paper/2407.21579/) (2024)
- [The FacT: Taming Latent Factor Models for Explainability with Factorization Trees](https://mlsanity.com/paper/1906.02037/) (2019)
- [Deep Reinforcement Learning-based Obstacle Avoidance for Robot Movement in Warehouse Environments](https://mlsanity.com/paper/2409.14972/) (2024)
- [NEVLP: Noise-Robust Framework for Efficient Vision-Language Pre-training](https://mlsanity.com/paper/2409.09582/) (2024)
- [Graphical Structural Learning of rs-fMRI data in Heavy Smokers](https://mlsanity.com/paper/2409.08395/) (2024)
- [Few-Shot Learning with Adaptive Weight Masking in Conditional GANs](https://mlsanity.com/paper/2412.03105/) (2024)

## Built on by (most cited first, 5 of 5)

- [Evaluating Supervised Learning Models for Fraud Detection: A Comparative Study of Classical and Deep Architectures on Imbalanced Transaction Data](https://mlsanity.com/paper/2505.22521/) (2025)
- [Reward-Shifted Speculative Sampling Is An Efficient Test-Time Weak-to-Strong Aligner](https://mlsanity.com/paper/2508.15044/) (2025)
- [How Do Agentic AI Systems Address Performance Optimizations? A BERTopic-Based Analysis of Pull Requests](https://mlsanity.com/paper/2512.24630/) (2025)
- [On the Stability of Prompt Ranking in Large Language Model Evaluation](https://mlsanity.com/paper/2606.24381/) (2026)
- [MaCP: Minimal yet Mighty Adaptation via Hierarchical Cosine Projection](https://mlsanity.com/paper/2505.23870/) (2025)

## Abstract (arXiv)

This paper presents a comprehensive performance evaluation of Large Language Models (LLMs) in solving programming challenges from Leetcode, a widely used platform for algorithm practice and technical interviews. We began by crawling the Leetcode website to collect a diverse set of problems encompassing various difficulty levels and topics. Using this dataset, we generated solutions with multiple LLMs, including GPT-4 and GPT-3.5-turbo (ChatGPT-turbo). The generated solutions were systematically evaluated for correctness and efficiency. We employed the pass@k metric to assess the success rates within a given number of attempts and analyzed the runtime performance of the solutions. Our results highlight the strengths and limitations of current LLMs [10] in code generation and problem-solving tasks, providing insights into their potential applications and areas for improvement in automated programming assistance.