mlsanityData to 14 Oct 2026

Worst-Case Symbolic Constraints Analysis and Generalisation with Large Language Models

arXiv 2506.08171 ↗First posted 9 Jun 2025cs.SE cs.AIcited by 1

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Large language models (LLMs) have demonstrated strong performance on coding tasks such as generation, completion and repair, but their ability to handle complex symbolic reasoning over code still remains underexplored. We introduce the task of worst-case symbolic constraints analysis, which requires inferring the symbolic constraints that characterise worst-case program executions; these constraints can be solved to obtain inputs that expose performance bottlenecks or denial-of-service vulnerabilities in software systems. We show that even state-of-the-art LLMs (e.g., GPT-5) struggle when applied directly on this task. To address this challenge, we propose WARP, an innovative neurosymbolic approach that computes worst-case constraints on smaller concrete input sizes using existing program analysis tools, and then leverages LLMs to generalise these constraints to larger input sizes. Concretely, WARP comprises: (1) an incremental strategy for LLM-based worst-case reasoning, (2) a solver-aligned neurosymbolic framework that integrates reinforcement learning with SMT (Satisfiability Modulo Theories) solving, and (3) a curated dataset of symbolic constraints. Experimental results show that WARP consistently improves performance on worst-case constraint reasoning. Leveraging the curated constraint dataset, we use reinforcement learning to fine-tune a model, WARP-1.0-3B, which significantly outperforms size-matched and even larger baselines. These results demonstrate that incremental constraint reasoning enhances LLMs' ability to handle symbolic reasoning and highlight the potential for deeper integration between neural learning and formal methods in rigorous program analysis.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 13 of its 27 references that are in the map. First posted on arXiv 9 Jun 2025.

Builds on

13 of 27 references.

References · 13

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  5. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  6. 2023LeanDojo: Theorem Proving with Retrieval-Augmented Language Modelscited by 427
  7. 2020Generative Language Modeling for Automated Theorem Provingcited by 388
  8. 2022Thor: Wielding Hammers to Integrate Language Models and Automated Theorem Proverscited by 98
  9. 2023ChatGPT for Vulnerability Detection, Classification, and Repair: How Far Are We?cited by 44
  10. 2023AI-assisted coding: Experiments with GPT-4cited by 43
  11. 2023A New Era in Software Security: Towards Self-Healing Software via Large Language Models and Formal Verificationcited by 42
  12. 2019Towards Finding Longer Proofscited by 10
1 more
  1. 2018Badger: Complexity Analysis with Fuzzing and Symbolic Executioncited by 4

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2025A Neurosymbolic Approach to Loop Invariant Generation via Weakest Precondition Reasoning
  2. 2025CodeCrash: Exposing LLM Fragility to Misleading Natural Language in Code Reasoning
  3. 2026From Errors to Proofs: Minimal-Core-Guided Repair for Neuro-Symbolic Constraint Solving
  4. 2026Exploring Extrinsic and Intrinsic Properties for Effective Reasoning with Code Interpreter
  5. 2025Breakpoint: Scalable evaluation of system-level reasoning in LLM code agents
  6. 2026Neuro-Symbolic Verification on Instruction Following of LLMs
  7. 2026NeuroProlog: Multi-Task Fine-Tuning for Neurosymbolic Mathematical Reasoning via the Cocktail Effect
  8. 2026Beyond Accuracy: Diagnosing Algebraic Reasoning Failures in LLMs Across Nine Complexity Dimensions