mlsanityData to 14 Oct 2026

ChLogic: Evaluating Robustness of Logical Reasoning in Chinese Expressions

arXiv 2606.17905 ↗First posted 16 Jun 2026cs.CLcited by 0

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Large language models perform increasingly well on standardized logical reasoning benchmarks, but whether this ability remains robust beyond English is unclear. We introduce ChLogic, an English--Chinese aligned benchmark that tests whether models preserve logical reasoning performance when the same latent logical structure is expressed in English and diverse Chinese surface realizations. Built from formal logical templates, the benchmark contains three data sets: (i) the General aligned set, derived from 60 General Propositions across nine template families; (ii) the Difficult aligned set, derived from 40 Difficult Problems; and (iii) the Chinese-only set, covering 15 language-specific phenomenon types. Each aligned item pairs one English reference expression with five Chinese realizations. Experiments on Qwen3, Ministral, and GLM models reveal a persistent English--Chinese performance gap. Back-translation from standard Chinese into English often improves performance on the General aligned set, but produces mixed effects on the Difficult aligned set, where Qwen3-32B and GLM-5.1 perform worse after translation. These results indicate that Chinese surface realization, translation artifacts, and model-specific behavior jointly affect multilingual logical reasoning. Overall, ChLogic provides a useful stress test for the robustness of multilingual reasoning.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 8 of its 26 references that are in the map. First posted on arXiv 16 Jun 2026.

Builds on

8 of 26 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026LogicSkills: A Structured Benchmark for Formal Reasoning in Large Language Models6 Feb
  2. 2025MultiNRC: A Challenging and Native Multilingual Reasoning Evaluation Benchmark for LLMs23 Jul
  3. 2024Reasoning Robustness of LLMs to Adversarial Typographical Errors8 Nov

Other references · 5

  1. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 708
  2. 2024GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Modelscited by 550
  3. 2015Towards AI-Complete Question Answering: A Set of Prerequisite Toy Taskscited by 314
  4. 2023SeaEval for Multilingual Foundation Models: From Cross-Lingual Alignment to Cultural Reasoningcited by 94
  5. 2022LogicInference: A New Dataset for Teaching Logical Inference to seq2seq Modelscited by 12

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Rethinking the Multilingual Reasoning Gap with Layer Swap
  2. 2024Benchmarking Chinese Commonsense Reasoning of LLMs: From Chinese-Specifics to Reasoning-Memorization Correlations
  3. 2025JustLogic: A Comprehensive Benchmark for Evaluating Deductive Reasoning in Large Language Models
  4. 2025A Comprehensive Evaluation of Multilingual Chain-of-Thought Reasoning: Performance, Consistency, and Faithfulness Across Languages
  5. 2025Cross-Lingual Consistency: A Novel Inference Framework for Advancing Reasoning in Large Language Models
  6. 2024Calibrating Reasoning in Language Models with Internal Consistency
  7. 2023Chain-of-Thought Hub: A Continuous Effort to Measure Large Language Models' Reasoning Performance
  8. 2025Why Do Multilingual Reasoning Gaps Emerge in Reasoning Language Models?