mlsanityData to 7 Oct 2026

Interpreting Multi-Attribute Confounding through Numerical Attributes in Large Language Models

arXiv 2511.04053 ↗First posted 6 Nov 2025cs.AIcited by 0

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Although behavioral studies have documented numerical reasoning errors in large language models (LLMs), the underlying representational mechanisms remain unclear. We hypothesize that numerical attributes occupy shared latent subspaces and investigate two questions:(1) How do LLMs internally integrate multiple numerical attributes of a single entity? (2)How does irrelevant numerical context perturb these representations and their downstream outputs? To address these questions, we combine linear probing with partial correlation analysis and prompt-based vulnerability tests across models of varying sizes. Our results show that LLMs encode real-world numerical correlations but tend to systematically amplify them. Moreover, irrelevant context induces consistent shifts in magnitude representations, with downstream effects that vary by model size. These findings reveal a vulnerability in LLM decision-making and lay the groundwork for fairer, representation-aware control under multi-attribute entanglement.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 22 of its 46 references that are in the map. First posted on arXiv 6 Nov 2025.

Builds on

22 of 46 references.

References · 22

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  3. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  4. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  5. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
  6. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
  7. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  8. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
  9. 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 333
  10. 2023In-Context Learning Creates Task Vectorscited by 302
  11. 2024Not All Language Model Features Are One-Dimensionally Linearcited by 201
  12. 2024The Semantic Hub Hypothesis: Language Models Share Semantic Representations Across Languages and Modalitiescited by 69
10 more
  1. 2025Language Models Use Trigonometry to Do Additioncited by 68
  2. 2024Evaluating LLMs' Mathematical Reasoning in Financial Document Question Answeringcited by 25
  3. 2024Language Models Do Hard Arithmetic Tasks Easily and Hardly Do Easy Arithmetic Taskscited by 24
  4. 2024Beyond Single Concept Vector: Modeling Concept Subspace in LLMs with Gaussian Distributioncited by 24
  5. 2021Do Language Models Know the Way to Rome?cited by 18
  6. 2023More Samples or More Prompts? Exploring Effective In-Context Sampling for LLM Few-Shot Prompt Engineeringcited by 13
  7. 2025CoME: An Unlearning-based Approach to Conflict-free Model Editingcited by 12
  8. 2024The Geometry of Numerical Reasoning: Language Models Compare Numeric Properties in Linear Subspacescited by 10
  9. 2025Related Knowledge Perturbation Matters: Rethinking Multiple Pieces of Knowledge Editing in Same-Subjectcited by 10
  10. 2024ChatGPT Doesn't Trust Chargers Fans: Guardrail Sensitivity in Contextcited by 9

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Knowing the Facts but Choosing the Shortcut: Understanding How Large Language Models Compare Entities
  2. 2026Large Reasoning Models Are (Not Yet) Multilingual Latent Reasoners
  3. 2025Large Language Models Encode Semantics and Alignment in Linearly Separable Representations
  4. 2025Discovering Hierarchical Latent Capabilities of Language Models via Causal Representation Learning
  5. 2026Unveiling the Reasoning Process of Large Language Models
  6. 2026Language Is an Insufficient Substrate for Quantitative Reasoning, and Consequential Domains Need Large Quantitative Models
  7. 2026Is my model perplexed for the right reason? Contrasting LLMs' Benchmark Behavior with Token-Level Perplexity
  8. 2026LLM Reasoning Is Latent, Not the Chain of Thought