# Benchmarking Large Language Models on Floating-Point Error Classification

arXiv: https://arxiv.org/abs/2606.31308 · HTML: https://mlsanity.com/paper/2606.31308/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 33 references that are in the map.
- First posted on arXiv 30 Jun 2026.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [gpt-oss-120b & gpt-oss-20b Model Card](https://mlsanity.com/paper/2508.10925/) (2025)
- [A Survey on Large Language Models for Code Generation](https://mlsanity.com/paper/2406.00515/) (2024)
- [Examining Zero-Shot Vulnerability Repair with Large Language Models](https://mlsanity.com/paper/2112.02125/) (2021)
- [A Survey on Large Language Models for Mathematical Reasoning](https://mlsanity.com/paper/2506.08446/) (2025)
- [A Survey of Large Language Model Agents for Question Answering](https://mlsanity.com/paper/2503.19213/) (2025)
- [A Systematic Literature Review on Large Language Models for Automated Program Repair](https://mlsanity.com/paper/2405.01466/) (2024)
- [A Survey of LLM-based Automated Program Repair: Taxonomies, Design Paradigms, and Applications](https://mlsanity.com/paper/2506.23749/) (2025)
- [Assembly of Experts: Linear-time construction of the Chimera LLM variants with emergent and adaptable behaviors](https://mlsanity.com/paper/2506.14794/) (2025)
- [Can LLMs Find Bugs in Code? An Evaluation from Beginner Errors to Security Vulnerabilities in Python and C++](https://mlsanity.com/paper/2508.16419/) (2025)
- [Finding Root Causes of Floating Point Error with Herbgrind](https://mlsanity.com/paper/1705.10416/) (2017)
- [LLM4FP: LLM-Based Program Generation for Triggering Floating-Point Inconsistencies Across Compilers](https://mlsanity.com/paper/2509.00256/) (2025)
- [Formal that "Floats" High: Formal Verification of Floating Point Arithmetic](https://mlsanity.com/paper/2512.06850/) (2025)
- [Assessing Large Language Models for Stabilizing Numerical Expressions in Scientific Software](https://mlsanity.com/paper/2604.04854/) (2026)

## Abstract (arXiv)

This paper investigates the capability of Large Language Models (LLMs) to detect and classify floating-point errors statically in software code. We introduce InterFLOPBench, a benchmark of 90 C kernels with 1 130 test samples designed to evaluate LLMs across six categories of floating-point error: cancellation, comparison, division by zero, overflow, underflow and NaN, compared across 14 LLMs. The evaluation framework treats floating-point error detection as a multi-label classification problem and employs the F1-score metric to measure performance. Results demonstrate that latest models (Qwen 3 32b, Gemini 2.5 Flash, Phi 4 Reasoning, DeepSeek R1T2, and gpt-oss 20b and 120b) achieve a performance greater than 0.88 overall F1-score. Performance varies between error categories, between explicit operations such as division by zero (Average F1-score: 0.8479) and more subtle numerical phenomena such as underflow (Average F1-score: 0.6059) and cancellation (Average F1-score: 0.6164).