# LogiCase: Effective Test Case Generation from Logical Description in Competitive Programming

arXiv: https://arxiv.org/abs/2505.15039 · HTML: https://mlsanity.com/paper/2505.15039/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 27 references that are in the map.
- First posted on arXiv 21 May 2025.

## Builds on (most cited first, 16 of 16)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [Competition-Level Code Generation with AlphaCode](https://mlsanity.com/paper/2203.07814/) (2022)
- [Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation](https://mlsanity.com/paper/2305.01210/) (2023)
- [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://mlsanity.com/paper/2002.08155/) (2020)
- [CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation](https://mlsanity.com/paper/2109.00859/) (2021)
- [UniXcoder: Unified Cross-Modal Pre-training for Code Representation](https://mlsanity.com/paper/2203.03850/) (2022)
- [CodeNet: A Large-Scale AI for Code Dataset for Learning a Diversity of Coding Tasks](https://mlsanity.com/paper/2105.12655/) (2021)
- [Fuzz4All: Universal Fuzzing with Large Language Models](https://mlsanity.com/paper/2308.04748/) (2023)
- [No More Manual Tests? Evaluating and Improving ChatGPT for Unit Test Generation](https://mlsanity.com/paper/2305.04207/) (2023)
- [Grammar Prompting for Domain-Specific Language Generation with Large Language Models](https://mlsanity.com/paper/2305.19234/) (2023)
- [Effective Test Generation Using Pre-trained Large Language Models and Mutation Testing](https://mlsanity.com/paper/2308.16557/) (2023)
- [TESTEVAL: Benchmarking Large Language Models for Test Case Generation](https://mlsanity.com/paper/2406.04531/) (2024)
- [RePair: Automated Program Repair with Process-based Feedback](https://mlsanity.com/paper/2408.11296/) (2024)
- [Predicting Patch Correctness Based on the Similarity of Failing Test Cases](https://mlsanity.com/paper/2107.13296/) (2021)

## Built on by (most cited first, 4 of 4)

- [AutoCode: LLMs as Problem Setters for Competitive Programming](https://mlsanity.com/paper/2510.12803/) (2025)
- [X-Coder: Advancing Competitive Programming with Synthetic Tasks, Solutions, and Tests](https://mlsanity.com/paper/2601.06953/) (2026)
- [SAGE:Specification-Aware Grammar Extraction for Automated Test Case Generation with LLMs](https://mlsanity.com/paper/2506.11081/) (2025)
- [Coding Agents as Test-Suite Auditors: Finding What Official Suites Miss While Approaching What They Catch](https://mlsanity.com/paper/2608.01715/) (2026)

## Abstract (arXiv)

Automated Test Case Generation (ATCG) is crucial for evaluating software reliability, particularly in competitive programming where robust algorithm assessments depend on diverse and accurate test cases. However, existing ATCG methods often fail to meet complex specifications or generate effective corner cases, limiting their utility. In this work, we introduce Context-Free Grammars with Counters (CCFGs), a formalism that captures both syntactic and semantic structures in input specifications. Using a fine-tuned CodeT5 model, we translate natural language input specifications into CCFGs, enabling the systematic generation of high-quality test cases. Experiments on the CodeContests dataset demonstrate that CCFG-based test cases outperform baseline methods in identifying incorrect algorithms, achieving significant gains in validity and effectiveness. Our approach provides a scalable and reliable grammar-driven framework for enhancing automated competitive programming evaluations.