mlsanityData to 14 Oct 2026

Lost in Reordering: Structural Sensitivity of Multilingual LLMs under Semantics-Preserving Perturbations

arXiv 2609.03511 ↗First posted 3 Sep 2026cs.CLcited by 0

On the map: LLM reasoning → Reasoning benchmarks and evaluation

also in 1 more
Abstract (arXiv)

Large Language Models (LLMs) demonstrate strong multilingual reasoning performance, yet their robustness to semantics-preserving structural variation remains underexplored, particularly for relatively free word-order languages. We investigate the structural sensitivity of multilingual LLMs using two linguistically grounded perturbation settings in Hindi and Malayalam: constrained constituent reordering and active-passive voice transformation. We introduce a benchmark dataset IndicReStruct, with two variants, GSM8K-Reordered and GSM8K-Voice, constructed from GSM8K while preserving semantic meaning. Across six state-of-the-art LLMs and multiple prompting strategies, we observe consistent and significant degradation in mathematical reasoning performance under structurally perturbed inputs. To further understand these failures, we perform qualitative error analysis and mechanistic interpretability experiments using residual-stream activation patching. Our analyses show that reasoning failures frequently arise from disruptions in entity-quantity alignment and that intermediate transformer layers contribute most strongly toward reasoning restoration. Overall, our findings suggest that current multilingual LLMs remain highly sensitive to surface syntactic realization and lack robust compositional invariance under structurally different but semantically equivalent inputs.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 18 of its 35 references that are in the map. First posted on arXiv 3 Sep 2026.

Builds on

18 of 35 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Exploring the Limitations of Large Language Models in Compositional Relation Reasoning5 Mar

Other references · 17

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  4. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  5. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  6. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  7. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,386
  8. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 560
  9. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 422
  10. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 359
  11. 2025MATH-Perturb: Benchmarking LLMs' Math Reasoning Abilities against Hard Perturbationscited by 89
  12. 2024Potential and Limitations of LLMs in Capturing Structured Semantics: A Case Study on SRLcited by 10
5 more
  1. 2024A Three-Pronged Approach to Cross-Lingual Adaptation with Multilingual LLMscited by 6
  2. 2026Same Meaning, Different Scores: Lexical and Syntactic Sensitivity in LLM Evaluationcited by 4
  3. 2025Mathematics Isn't Culture-Free: Probing Cultural Gaps via Entity and Scenario Perturbationscited by 3
  4. 2025Semantically Cohesive Word Grouping in Indian Languagescited by 1
  5. 2025BHRAM-IL: A Benchmark for Hallucination Recognition and Assessment in Multiple Indian Languagescited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Rethinking the Multilingual Reasoning Gap with Layer Swap
  2. 2025Cross-Lingual Consistency: A Novel Inference Framework for Advancing Reasoning in Large Language Models
  3. 2026Fragile Reasoning: A Mechanistic Analysis of LLM Sensitivity to Meaning-Preserving Perturbations
  4. 2025The Hidden Space of Safety: Understanding Preference-Tuned LLMs in Multilingual context
  5. 2025Exploring LLM Reasoning Through Controlled Prompt Variations
  6. 2025Multidimensional Consistency Improves Reasoning in Language Models
  7. 2025Evaluating Multilingual and Code-Switched Alignment in LLMs via Synthetic Natural Language Inference
  8. 2025The Unreasonable Effectiveness of Model Merging for Cross-Lingual Transfer in LLMs