mlsanityData to 7 Oct 2026

Large Language Models Are Cross-Lingual Knowledge-Free Reasoners

arXiv 2406.16655 ↗First posted 24 Jun 2024cs.CLcited by 16

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Large Language Models have demonstrated impressive reasoning capabilities across multiple languages. However, the relationship between capabilities in different languages is less explored. In this work, we decompose the process of reasoning tasks into two separated components: knowledge retrieval and knowledge-free reasoning, and analyze the relationship between cross-lingual transferability and these two components. With adapted commonsense reasoning datasets and constructed knowledge-free reasoning datasets, we show that the knowledge-free reasoning capability can be nearly perfectly transferred across various source-target language directions despite the secondary impact of resource in some specific target languages, while cross-lingual knowledge retrieval significantly hinders the transfer. Moreover, by analyzing the hidden states and feed-forward network neuron activation during the reasoning, we show that higher similarity of hidden representations and larger overlap of activated neurons could explain the better cross-lingual transferability of knowledge-free reasoning than knowledge retrieval. Thus, we hypothesize that knowledge-free reasoning shares similar neurons in different languages for reasoning, while knowledge is stored separately in different languages. Our code and data is available at: https://github.com/NJUNLP/Knowledge-Free-Reasoning.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 31 of its 51 references that are in the map. First posted on arXiv 24 Jun 2024.

Builds on

31 of 51 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Large Language Models are Limited in Out-of-Context Knowledge Reasoning11 Jun
  2. 2024Question Translation Training for Better Multilingual Reasoning15 Jan
  3. 2022Towards Reasoning in Large Language Models: A Survey20 Dec

Other references · 28

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  6. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  7. 2023Qwen Technical Reportcited by 3,273
  8. 2023Mistral 7Bcited by 3,065
  9. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  10. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  11. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
  12. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 518
16 more
  1. 2021A Diverse Corpus for Evaluating and Developing English Math Word Problem Solverscited by 314
  2. 2020ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Languagecited by 309
  3. 2024Do Llamas Work in English? On the Latent Language of Multilingual Transformerscited by 275
  4. 2024How do Large Language Models Handle Multilingualism?cited by 148
  5. 2021Composable Sparse Fine-Tuning for Cross-Lingual Transfercited by 144
  6. 2023A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasetscited by 123
  7. 2023Cross-Lingual Consistency of Factual Knowledge in Multilingual Language Modelscited by 117
  8. 2024Do language models plan ahead for future tokens?cited by 47
  9. 2023Language Versatilists vs. Specialists: An Empirical Revisiting on Multilingual Transfer Abilitycited by 27
  10. 2024Multilingual Pretraining and Instruction Tuning Improve Cross-Lingual Knowledge Alignment, But Only Shallowlycited by 27
  11. 2022Crosslingual Generalization through Multitask Finetuningcited by 26
  12. 2021MTG: A Benchmark Suite for Multilingual Text Generationcited by 22
  13. 2024An Investigation of Neuron Activation as a Unified Lens to Explain Chain-of-Thought Eliciting Arithmetic Reasoning of LLMscited by 20
  14. 2024LLMCheckup: Conversational Examination of Large Language Models via Interpretability Tools and Self-Explanationscited by 16
  15. 2024SambaLingo: Teaching Large Language Models New Languagescited by 11
  16. 2024What Is Missing in Multilingual Visual Reasoning and How to Fix Itcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 6; 2026: 9.

16 papers.

Sort:

By area:

  1. 2025MPO: Multilingual Safety Alignment via Reward Gap Optimizationcited by 18
  2. 2024Benchmarking and Improving Large Vision-Language Models for Fundamental Visual Graph Understanding and Reasoningcited by 17
  3. 2025How Does Alignment Enhance LLMs' Multilingual Capabilities? A Language Neurons Perspectivecited by 8
  4. 2025Cross-Lingual Transfer of Cultural Knowledge: An Asymmetric Phenomenoncited by 5
  5. 2025Understanding LLMs' Cross-Lingual Context Retrieval: How Good It Is And Where It Comes Fromcited by 4
  6. 2025MMA-ASIA: A Multilingual and Multimodal Alignment Framework for Culturally-Grounded Evaluationcited by 4
  7. 2026Cross-Lingual Activation Steering for Multilingual Language Modelscited by 4
  8. 2025Multilingual Knowledge Graph Completion via Efficient Multilingual Knowledge Sharingcited by 2
  9. 2026Beyond Language: Format-Agnostic Reasoning Subspaces in Large Language Modelscited by 1
  10. 2026On the Robustness of Knowledge Editing for Detoxificationcited by 0
  11. 2026The Periodic Table of LLM Reasoning: A Structured Survey of Reasoning Paradigms, Methods, and Failure Modescited by 0
  12. 2026Are Multilingual Models Actually Improving? Isolating True Cross-Lingual Transfercited by 0
4 more
  1. 2026Bridging the English-Arabic Medical Knowledge Gap: Targeted Low-Rank Adaptation via Causal Layer Selectioncited by 0
  2. 2026Discovering Cross-Language Reasoning Invariance in LLMs with Geometry-Invariant Sparse Autoencoderscited by 0
  3. 2026Cross Lingual Transfer in Tulu Legal Comprehension: Script-Dependent Improvement and RAG-Induced Knowledge Conflictcited by 0
  4. 2026Cross-Lingual Alignment for Decoder-Only Models using MoE Routerscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025When Less Language is More: Language-Reasoning Disentanglement Makes LLMs Better Multilingual Reasoners
  2. 2023Breaking the Language Barrier: Improving Cross-Lingual Reasoning with Structured Self-Attention
  3. 2025Language-Specific Latent Process Hinders Cross-Lingual Performance
  4. 2026Rethinking the Multilingual Reasoning Gap with Layer Swap
  5. 2024Disentangling Memory and Reasoning Ability in Large Language Models
  6. 2025Language Matters: How Do Multilingual Input and Reasoning Paths Affect Large Reasoning Models?
  7. 2026Large Reasoning Models Are (Not Yet) Multilingual Latent Reasoners
  8. 2024Unveiling Factual Recall Behaviors of Large Language Models through Knowledge Neurons