mlsanityData to 14 Oct 2026

Adversarial Concept Search: Predicting Compositional Errors From Feature Geometry

arXiv 2606.13934 ↗First posted 11 Jun 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Humans cannot always intuit what scenarios are most challenging to LLMs. Hoping to capture challenging edge cases, developers either design problems to be difficult for humans or curate extensive benchmarks. What if we could instead anticipate which scenarios a model will fail on? In this paper, we use an LLM's representational geometry to predict which concept combinations it will fail on. We attribute this compositional failure to interference between salient features. In tasks that require systematic composition - toy programmatic settings, multihop reasoning, multilingual factual recall - we find that when a pair of concepts is encoded near-orthogonally, the model reliably composes them. When their linear encodings are close, producing interference, the model fails to compose them. Our method reliably anticipates failure modes across different compositional tasks, without evaluating specific inputs. These results lay the groundwork to use representational geometry to identify high-risk examples, construct targeted stress tests, and provide a scalable foundation for active learning in real-world deployment.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 32 of its 49 references that are in the map. First posted on arXiv 11 Jun 2026.

Builds on

32 of 49 references.

References · 32

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2022Generative Adversarial Networkscited by 5,627
  3. 2013Intriguing properties of neural networkscited by 5,155
  4. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  5. 2022Toy Models of Superpositioncited by 971
  6. 2017Poincar'e Embeddings for Learning Hierarchical Representationscited by 684
  7. 2023In-Context Learning Creates Task Vectorscited by 312
  8. 2024Do Llamas Work in English? On the Latent Language of Multilingual Transformerscited by 281
  9. 2019Compositionality decomposed: how do neural networks generalise?cited by 265
  10. 2016Linear Algebraic Structure of Word Senses, with Applications to Polysemycited by 165
  11. 2024How do Large Language Models Handle Multilingualism?cited by 150
  12. 2022The Geometry of Multilingual Language Model Representationscited by 95
20 more
  1. 2025Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Modelscited by 73
  2. 2022Linear Connectivity Reveals Generalization Strategiescited by 56
  3. 2026When Models Manipulate Manifolds: The Geometry of a Counting Taskcited by 46
  4. 2025Lost in Multilinguality: Dissecting Cross-lingual Factual Inconsistency in Transformer Language Modelscited by 44
  5. 2025Paths Not Taken: Understanding and Mending the Multilingual Factual Recall Pipelinecited by 21
  6. 2026Subliminal Effects in Your Data: A General Mechanism via Log-Linearitycited by 19
  7. 2025Does Data Scaling Lead to Visual Compositional Generalization?cited by 13
  8. 2025Adversarial Examples Are Not Bugs, They Are Superpositioncited by 13
  9. 2026How Many Features Can a Language Model Store Under the Linear Representation Hypothesis?cited by 12
  10. 2025Internal Causal Mechanisms Robustly Predict Language Model Out-of-Distribution Behaviorscited by 11
  11. 2026From Data Statistics to Feature Geometry: How Correlations Shape Superpositioncited by 10
  12. 2025Feature Learning beyond the Lazy-Rich Dichotomy: Insights from Representational Geometrycited by 9
  13. 2026Align Once, Benefit Multilingually: Enforcing Multilingual Consistency for LLM Safety Alignmentcited by 9
  14. 2025Can Interpretation Predict Behavior on Unseen Data?cited by 8
  15. 2025How Do Language Models Compose Functions?cited by 8
  16. 2026Stop Probing, Start Coding: Why Linear Probes and Sparse Autoencoders Fail at Compositional Generalisationcited by 6
  17. 2025Are Knowledge and Reference in Multilingual Language Models Cross-Lingually Consistent?cited by 5
  18. 2025Beyond the Rosetta Stone: Unification Forces in Generalization Dynamicscited by 3
  19. 2026Representational Homomorphism Predicts and Improves Compositional Generalization In Transformer Language Modelcited by 2
  20. 2026Diagnosing Generalization Failures from Representational Geometry Markerscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026The Geometry of Representational Failures in Vision Language Models
  2. 2026Measuring Representation Robustness in Large Language Models for Geometry
  3. 2024Do Large Language Models Have Compositional Ability? An Investigation into Limitations and Scalability
  4. 2026Diagnosing the Sources of Compositional Failure in Vision-Language Models: A Controlled Analysis
  5. 2023How Do In-Context Examples Affect Compositional Generalization?
  6. 2026Geometric Deviation as an Unsupervised Pre-Generation Reliability Signal: Probing LLM Representations for Answerability
  7. 2025Concept Concentration for Faithful Representation Intervention
  8. 2026Lingtai: What Concept Geometry Reveals--and Does Not Reveal--About LLM Inference