mlsanityData to 14 Oct 2026

Towards Effective In-context Cross-domain Knowledge Transfer via Domain-invariant-neurons-based Retrieval

arXiv 2604.05383 ↗First posted 7 Apr 2026cs.AIcited by 1

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Large language models (LLMs) have made notable progress in logical reasoning, yet still fall short of human-level performance. Current boosting strategies rely on expert-crafted in-domain demonstrations, limiting their applicability in expertise-scarce domains, such as specialized mathematical reasoning, formal logic, or legal analysis. In this work, we demonstrate the feasibility of leveraging cross-domain demonstrating examples to boost the LLMs' reasoning performance. Despite substantial domain differences, many reusable implicit logical structures are shared across domains. In order to effectively retrieve cross-domain examples for unseen domains under investigation, in this work, we further propose an effective retrieval method, called domain-invariant neurons-based retrieval (DIN-Retrieval). Concisely, DIN-Retrieval first summarizes a hidden representation that is universal across different domains. Then, during the inference stage, we use the DIN vector to retrieve structurally compatible cross-domain demonstrations for the in-context learning. Experimental results in multiple settings for the transfer of mathematical and logical reasoning demonstrate that our method achieves an average improvement of 1.8 over the state-of-the-art methods footnoteOur implementation is available at https://github.com/Leon221220/DIN-Retrieval.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 35 of its 55 references that are in the map. First posted on arXiv 7 Apr 2026.

Builds on

35 of 55 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025ProtoReasoning: Prototypes as the Foundation for Generalizable Reasoning in LLMs18 Jun
  2. 2022Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought3 Oct

Other references · 33

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  5. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  6. 2015Domain-Adversarial Training of Neural Networkscited by 3,044
  7. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  8. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
  9. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,392
  10. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  11. 2015Learning Transferable Features with Deep Adaptation Networkscited by 1,329
  12. 2020Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuningcited by 671
21 more
  1. 2021Learning To Retrieve Prompts for In-Context Learningcited by 648
  2. 2021Knowledge Neurons in Pretrained Transformerscited by 601
  3. 2020A Brief Review of Domain Adaptationcited by 288
  4. 2019The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectivescited by 139
  5. 2021Deep Subdomain Adaptation Network for Image Classificationcited by 119
  6. 2021Neuron-level Interpretation of Deep NLP Models: A Surveycited by 81
  7. 2025Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspectivecited by 76
  8. 2023Large Language Models Can be Lazy Learners: Analyze Shortcuts in In-Context Learningcited by 69
  9. 2024Demystifying Chains, Trees, and Graphs of Thoughtscited by 63
  10. 2021On the Pitfalls of Analyzing Individual Neurons in Language Modelscited by 62
  11. 2023Coverage-based Example Selection for In-Context Learningcited by 59
  12. 2024Confidence Regulation Neurons in Language Modelscited by 57
  13. 2023Explore Spurious Correlations at the Concept Level in Language Models for Text Classificationcited by 48
  14. 2024Uncertainty Quantification for In-Context Learning of Large Language Modelscited by 45
  15. 2020Domain Conditioned Adaptation Networkcited by 43
  16. 2024Examining the robustness of LLM evaluation to the distributional assumptions of benchmarkscited by 31
  17. 2023In-context Learning Generalizes, But Not Always Robustly: The Case of Syntaxcited by 22
  18. 2024Identifying Query-Relevant Neurons in Large Language Models for Long-Form Textscited by 14
  19. 2024Unlocking the Power of LLM Uncertainty for Active In-Context Example Selectioncited by 9
  20. 2025From Neurons to Semantics: Evaluating Cross-Linguistic Alignment Capabilities of Large Language Models via Neurons Alignmentcited by 9
  21. 2024Adaptation Odyssey in LLMs: Why Does Additional Pretraining Sometimes Fail to Improve?cited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026CoDA: Towards Effective Cross-domain Knowledge Transfer via CoT-guided Domain Adaptationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Rethinking Reasoning in LLMs: Neuro-Symbolic Local RetoMaton Beyond ICL and CoT
  2. 2025RARE: Retrieval-Augmented Reasoning Modeling
  3. 2024Large Language Models Are Cross-Lingual Knowledge-Free Reasoners
  4. 2025Directional Attractors in LLM Reasoning: How Similarity Retrieval Steers Iterative Summarization Based Reasoning
  5. 2023Knowledge Solver: Teaching LLMs to Search for Domain Knowledge from Knowledge Graphs
  6. 2026Identifying and Transferring Reasoning-Critical Neurons: Improving LLM Inference Reliability via Activation Steering
  7. 2026Neuron Level Analysis of Large Language Model in Legal Domain Reasoning
  8. 2026Discovering a Shared Logical Subspace: Steering LLM Logical Reasoning via Alignment of Natural-Language and Symbolic Views