mlsanityData to 14 Oct 2026

BANGLAPLAN: A Benchmark for Constraint-Based Planning and Decision-Making in Bangla

arXiv 2610.19704 ↗First posted 14 Oct 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) increasingly help users arrange appointments, plan journeys, and allocate limited resources. These tasks require models to satisfy constraints on time, cost, capacity, and order, or determine that no feasible solution exists. Bangla is the world's seventh most widely spoken language, and existing benchmarks cover many language and reasoning tasks. However, benchmarks for evaluating planning and decision-making in Bangla under explicit constraints remain lacking. We introduce BANGLAPLAN, a benchmark of 1,000 problems across ten task families grounded in everyday life and institutions in Bangladesh. It includes feasible and infeasible problems in four language registers: everyday, formal, colloquial, and Bangla--English mixed. Answers cover eight structured types, including schedules, routes, allocations, and ordered steps, in a common JSON format. We construct the benchmark through language-model-assisted drafting and independent human validation. We evaluate 11 open-weight and proprietary model configurations using exact match, separately reporting feasible-case accuracy, infeasibility detection, and format errors. We find a large performance gap: GPT-6.1 Sol achieves 98.7% accuracy and Qwen3.5-27B with thinking reaches 80.3%, while direct-answer configurations score at most 30.9%. Seven of eleven configurations perform below the always-``infeasible'' baseline. Our findings also suggest that harder problems primarily reduce plan-construction accuracy, while infeasibility detection remains more robust. BANGLAPLAN provides a reproducible basis for evaluating planning and decision-making in Bangla. BANGLAPLAN is available at: https://github.com/mrayhanulmasud/banglaplan.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 25 references that are in the map. First posted on arXiv 14 Oct 2026.

Builds on

18 of 25 references.

References · 18

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2024TravelPlanner: A Benchmark for Real-World Planning with Language Agentscited by 392
  3. 2022PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Changecited by 382
  4. 2024NATURAL PLAN: Benchmarking LLMs on Natural Language Planningcited by 128
  5. 2024INCLUDE: Evaluating Multilingual Language Understanding with Regional Knowledgecited by 86
  6. 2025VoiceAgentBench: Are Voice Assistants ready for agentic tasks?cited by 19
  7. 2024ChinaTravel: An Open-Ended Travel Planning Benchmark with Compositional Constraint Validation for Language Agentscited by 16
  8. 2025TigerLLM - A Family of Bangla Large Language Modelscited by 16
  9. 2025Script Fragmentation and Format: What Drives the English-Bengali Performance Gap in Open LLMs?cited by 9
  10. 2025BnMMLU: Measuring Massive Multitask Language Understanding in Bengalicited by 7
  11. 2026ConstraintBench: Benchmarking LLM Constraint Reasoning on Direct Optimizationcited by 7
  12. 2025Ticket-Bench: A Kickoff for Multilingual and Regionalized Agent Evaluationcited by 5
6 more
  1. 2026NCTB-QA: A Large-Scale Bangla Educational Question Answering Dataset and Benchmarking Performancecited by 2
  2. 2026GSM-Plus-BN: A Perturbation-Based Benchmark for Bangla Mathematical Reasoning in Large Language Modelscited by 2
  3. 2026TREK: A Travel Reasoning and Evaluation Kit for LLM Agents in Complex Trip Planningcited by 2
  4. 2026BanglaSocialBench: A Benchmark for Evaluating Sociopragmatic and Cultural Alignment of LLMs in Bangladeshi Social Interactioncited by 1
  5. 2026An Actionable Diagnosis of Multilingual, Multi-Agent Planning Failurescited by 1
  6. 2026WorldBench: Culturally Grounded Benchmark for Multilingual Agentscited by 1

Built on by

Not cited yet.