mlsanityData to 7 Oct 2026

Competition for attention predicts good-to-bad tipping in AI

arXiv 2602.14370 ↗First posted 16 Feb 2026cs.AI physics.app-ph physics.soc-phcited by 0

Not on the map. Why?

Abstract (arXiv)

More than half the global population now carries devices that can run ChatGPT-like language models with no Internet connection and minimal safety oversight -- and hence the potential to promote self-harm, financial losses and extremism among other dangers. Existing safety tools either require cloud connectivity or discover failures only after harm has occurred. Here we show that a large class of potentially dangerous tipping originates at the atomistic scale in such edge AI due to competition for the machinery's attention. This yields a mathematical formula for the dynamical tipping point n*, governed by dot-product competition for attention between the conversation's context and competing output basins, that reveals new control levers. Validated against multiple AI models, the mechanism can be instantiated for different definitions of 'good' and 'bad' and hence in principle applies across domains (e.g. health, law, finance, defense), changing legal landscapes (e.g. EU, UK, US and state level), languages, and cultural settings.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 25 of its 53 references that are in the map. First posted on arXiv 16 Feb 2026.

Builds on

25 of 53 references.

References · 25

  1. 2017Attention Is All You Needcited by 59,932
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  6. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  7. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  8. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  9. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  10. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  11. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  12. 2022Red Teaming Language Models with Language Modelscited by 983
13 more
  1. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  2. 2022In-context Learning and Induction Headscited by 930
  3. 2023Progress measures for grokking via mechanistic interpretabilitycited by 822
  4. 2020A Primer in BERTology: What we know about how BERT workscited by 807
  5. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 770
  6. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  7. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  8. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 700
  9. 2017Supervised Learning of Universal Sentence Representations from Natural Language Inference Datacited by 428
  10. 2023A Survey of Hallucination in Large Foundation Modelscited by 365
  11. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 313
  12. 2024How to use and interpret activation patchingcited by 191
  13. 2022Transformer Language Models without Positional Encodings Still Learn Positional Informationcited by 148

Built on by

Not cited yet.