Retriv at BLP-2025 Task 2: Test-Driven Feedback-Guided Framework for Bangla-to-Python Code Generation
Not on the map. Why?
Abstract (arXiv)
Large Language Models (LLMs) have advanced the automated generation of code from natural language prompts. However, low-resource languages (LRLs) like Bangla remain underrepresented due to the limited availability of instruction-to-code datasets and evaluation benchmarks. To address this, the BLP Workshop at IJCNLP-AACL 2025 introduced a shared task on "Code Generation in Bangla". In this work, we propose a method that combines instruction prompting with a test-driven, feedback-guided iterative refinement process using a fine-tuned Qwen2.5-14B model. The model generates code from Bangla instructions, tests it against unit tests, and iteratively refines any failing outputs through three evaluation passes, using test feedback to guide each step. This approach helped our team "Retriv" to secure 2nd place in the shared task with a Pass@1 score of 0.934. The analysis highlights challenges in Bangla instruction understanding and Python code generation, emphasizing the need for targeted methods in LRLs. We made experimental scripts publicly available for the community.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 7 of its 42 references that are in the map. First posted on arXiv 10 Nov 2025.
Builds on
7 of 42 references.
References · 7
- 2021Program Synthesis with Large Language Modelscited by 3,637
- 2023Teaching Large Language Models to Self-Debugcited by 988
- 2024Debug like a Human: A Large Language Model Debugger via Verifying Runtime Execution Step-by-stepcited by 127
- 2024mHumanEval -- A Multilingual Benchmark to Evaluate Large Language Models for Code Generationcited by 32
- 2025TigerLLM - A Family of Bangla Large Language Modelscited by 16
- 2025TituLLMs: A Family of Bangla LLMs with Comprehensive Benchmarkingcited by 9
- 2025TigerCoder: A Novel Suite of LLMs for Code Generation in Banglacited by 6
Built on by
Not cited yet.