mlsanityData to 7 Oct 2026

Python is Not Always the Best Choice: Embracing Multilingual Program of Thoughts

arXiv 2402.10691 ↗First posted 16 Feb 2024cs.CLcited by 8

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Program of Thoughts (PoT) is an approach characterized by its executable intermediate steps, which ensure the accuracy of the logical calculations in the reasoning process. Currently, PoT primarily uses Python. However, relying solely on a single language may result in suboptimal solutions and overlook the potential benefits of other programming languages. In this paper, we conduct comprehensive experiments on the programming languages used in PoT and find that no single language consistently delivers optimal performance across all tasks and models. The effectiveness of each language varies depending on the specific scenarios. Inspired by this, we propose a task and model agnostic approach called MultiPoT, which harnesses strength and diversity from various languages. Experimental results reveal that it significantly outperforms Python Self-Consistency. Furthermore, it achieves comparable or superior performance compared to the best monolingual PoT in almost all tasks across all models. In particular, MultiPoT achieves more than 4.6% improvement on average on ChatGPT (gpt-3.5-turbo-0701).

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 20 of its 39 references that are in the map. First posted on arXiv 16 Feb 2024.

Builds on

20 of 39 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Code Llama: Open Foundation Models for Code24 Aug

Other references · 19

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  4. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  5. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  6. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  7. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
  8. 2023Large Language Models as Optimizerscited by 830
  9. 2023StarCoder: may the source be with you!cited by 790
  10. 2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 591
  11. 2024Executable Code Actions Elicit Better LLM Agentscited by 537
  12. 2022The Stack: 3 TB of permissively licensed source codecited by 354
7 more
  1. 2022Text and Patterns: For Effective Chain of Thought, It Takes Two to Tangocited by 128
  2. 2023InferFix: End-to-End Program Repair with LLMscited by 113
  3. 2023Learning Performance-Improving Code Editscited by 103
  4. 2024If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agentscited by 77
  5. 2023Understanding the Effectiveness of Large Language Models in Detecting Security Vulnerabilitiescited by 58
  6. 2023BizBench: A Quantitative Reasoning Benchmark for Business and Financecited by 28
  7. 2023Exploring Equation as a Better Intermediate Meaning Representation for Numerical Reasoningcited by 7

Built on by

Cited by year

Citing papers per year: 2024: 4; 2025: 4.

8 papers.

  1. 2024A Survey on LLM-based Code Generation for Low-Resource and Domain-Specific Programming Languagescited by 65
  2. 2025A Survey of Multilingual Reasoning in Language Modelscited by 31
  3. 2024Search, Verify and Feedback: Towards Next Generation Post-training Paradigm of Foundation Models via Verifier Engineeringcited by 16
  4. 2025ChartMaster: Advancing Chart-to-Code Generation with Real-World Charts and Chart Similarity Reinforcement Learningcited by 15
  5. 2024FLEXTAF: Enhancing Table Reasoning with Flexible Tabular Formatscited by 8
  6. 2024In-Context Transfer Learning: Demonstration Synthesis by Transferring Similar Taskscited by 3
  7. 2025Is Compression Really Linear with Code Intelligence?cited by 2
  8. 2025Format-Adapter: Improving Reasoning Capability of LLMs by Adapting Suitable Formatcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024MultiLingPoT: Enhancing Mathematical Reasoning with Multilingual Program Fine-tuning
  2. 2025Towards Better Understanding of Program-of-Thought Reasoning in Cross-Lingual and Multilingual Environments
  3. 2024Eliciting Better Multilingual Structured Reasoning from LLMs through Code
  4. 2024mCoT: Multilingual Instruction Tuning for Reasoning Consistency in Language Models
  5. 2026Rethinking the Multilingual Reasoning Gap with Layer Swap
  6. 2025EfficientXLang: Towards Improving Token Efficiency Through Cross-Lingual Reasoning
  7. 2025A Comprehensive Evaluation of Multilingual Chain-of-Thought Reasoning: Performance, Consistency, and Faithfulness Across Languages
  8. 2025Language Matters: How Do Multilingual Input and Reasoning Paths Affect Large Reasoning Models?