mlsanityData to 13 Oct 2026

LogiCase: Effective Test Case Generation from Logical Description in Competitive Programming

arXiv 2505.15039 ↗First posted 21 May 2025cs.SE cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

Automated Test Case Generation (ATCG) is crucial for evaluating software reliability, particularly in competitive programming where robust algorithm assessments depend on diverse and accurate test cases. However, existing ATCG methods often fail to meet complex specifications or generate effective corner cases, limiting their utility. In this work, we introduce Context-Free Grammars with Counters (CCFGs), a formalism that captures both syntactic and semantic structures in input specifications. Using a fine-tuned CodeT5 model, we translate natural language input specifications into CCFGs, enabling the systematic generation of high-quality test cases. Experiments on the CodeContests dataset demonstrate that CCFG-based test cases outperform baseline methods in identifying incorrect algorithms, achieving significant gains in validity and effectiveness. Our approach provides a scalable and reliable grammar-driven framework for enhancing automated competitive programming evaluations.

Key facts. Cited by 4 papers in the map (data through 13 Oct 2026). Builds on 16 of its 27 references that are in the map. First posted on arXiv 21 May 2025.

Builds on

16 of 27 references.

References · 16

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,376
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,205
  3. 2023Code Llama: Open Foundation Models for Codecited by 2,130
  4. 2022Competition-Level Code Generation with AlphaCodecited by 1,445
  5. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,362
  6. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,304
  7. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 905
  8. 2022UniXcoder: Unified Cross-Modal Pre-training for Code Representationcited by 360
  9. 2021CodeNet: A Large-Scale AI for Code Dataset for Learning a Diversity of Coding Taskscited by 199
  10. 2023Fuzz4All: Universal Fuzzing with Large Language Modelscited by 88
  11. 2023No More Manual Tests? Evaluating and Improving ChatGPT for Unit Test Generationcited by 82
  12. 2023Grammar Prompting for Domain-Specific Language Generation with Large Language Modelscited by 58
4 more
  1. 2023Effective Test Generation Using Pre-trained Large Language Models and Mutation Testingcited by 51
  2. 2024TESTEVAL: Benchmarking Large Language Models for Test Case Generationcited by 38
  3. 2024RePair: Automated Program Repair with Process-based Feedbackcited by 16
  4. 2021Predicting Patch Correctness Based on the Similarity of Failing Test Casescited by 4

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025AutoCode: LLMs as Problem Setters for Competitive Programmingcited by 11
  2. 2025SAGE:Specification-Aware Grammar Extraction for Automated Test Case Generation with LLMscited by 0
  3. 2026X-Coder: Advancing Competitive Programming with Synthetic Tasks, Solutions, and Testscited by 0
  4. 2026Coding Agents as Test-Suite Auditors: Finding What Official Suites Miss While Approaching What They Catchcited by 0