mlsanityData to 14 Oct 2026

Concept Attractors in LLMs and their Applications

arXiv 2601.11575 ↗First posted 30 Dec 2025cs.CL cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) often map semantically related prompts to similar internal representations at specific layers, even when their surface forms differ widely. We show that this behavior can be explained through Iterated Function Systems (IFS), where layers act as contractive mappings toward concept-specific Attractors. We leverage this insight and develop simple, training-free methods that operate directly on these Attractors to solve a wide range of practical tasks, including language translation, hallucination reduction, guardrailing, and synthetic data generation. Despite their simplicity, these Attractor-based interventions match or exceed specialized baselines, offering an efficient alternative to heavy fine-tuning, generalizable in scenarios where baselines underperform.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 53 of its 73 references that are in the map. First posted on arXiv 30 Dec 2025.

Builds on

53 of 73 references.

References · 53

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  5. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  6. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  7. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
  8. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  9. 2024Qwen2 Technical Reportcited by 2,120
  10. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
  11. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  12. 2018World Modelscited by 1,513
38 more
  1. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
  2. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decodingcited by 688
  3. 2018Object Hallucination in Image Captioningcited by 669
  4. 2024TOFU: A Task of Fictitious Unlearning for LLMscited by 533
  5. 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearningcited by 504
  6. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 486
  7. 2023SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generationcited by 386
  8. 2023Who's Harry Potter? Approximate Unlearning in LLMscited by 381
  9. 2023Chameleon: Plug-and-Play Compositional Reasoning with Large Language Modelscited by 356
  10. 2023In-Context Learning Creates Task Vectorscited by 312
  11. 2023In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steeringcited by 245
  12. 2024On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Surveycited by 244
  13. 2023Fine-tuning Language Models for Factualitycited by 221
  14. 2023Model Sparsity Can Simplify Machine Unlearningcited by 214
  15. 2024Multi-Modal Hallucination Control by Visual Information Groundingcited by 208
  16. 2024Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMscited by 179
  17. 2017ParlAI: A Dialog Research Software Platformcited by 139
  18. 2024Towards Safer Large Language Models through Machine Unlearningcited by 139
  19. 2023Boundary Unlearningcited by 134
  20. 2024Large Language Model Unlearning via Embedding-Corrupted Promptscited by 116
  21. 2022Deep Unlearning via Randomized Conditionally Independent Hessianscited by 84
  22. 2024Transformers represent belief state geometry in their residual streamcited by 77
  23. 2018Mad Max: Affine Spline Insights into Deep Learningcited by 55
  24. 2023ONCE: Boosting Content-based Recommendation with Both Open- and Closed-source Large Language Modelscited by 54
  25. 2024Don't Miss the Forest for the Trees: Attentional Vision Calibration for Large Vision Language Modelscited by 47
  26. 2024Understanding and Patching Compositional Reasoning in LLMscited by 46
  27. 2023Large Language Models for Code Analysis: Do LLMs Really Do Their Job?cited by 45
  28. 2024Jailbreaking Large Language Models Against Moderation Guardrails via Cipher Characterscited by 42
  29. 2023Chainpoll: A high efficacy method for LLM hallucination detectioncited by 39
  30. 2024Steering Large Language Models using Conceptors: Improving Addition-Based Activation Engineeringcited by 39
  31. 2024Better Synthetic Data by Retrieving and Transforming Existing Datasetscited by 36
  32. 2024Does Representation Matter? Exploring Intermediate Layers in Large Language Modelscited by 34
  33. 2023Forgetting before Learning: Utilizing Parametric Arithmetic for Knowledge Updating in Large Language Modelscited by 15
  34. 2024Verified Code Transpilation with LLMscited by 15
  35. 2025Unveiling Attractor Cycles in Large Language Models: A Dynamical Systems View of Successive Paraphrasingcited by 15
  36. 2024Explaining Code with a Purpose: An Integrated Approach for Developing Code Comprehension and Prompting Skillscited by 11
  37. 2025Transformer Dynamics: A neuroscientific approach to interpretability of large language modelscited by 11
  38. 2024DESTEIN: Navigating Detoxification of Language Models via Universal Steering Pairs and Head-wise Activation Fusioncited by 9

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Identity as Attractor: Geometric Evidence for Persistent Agent Architecture in LLM Activation Spacecited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2026Solve the Loop: Attractor Models for Language and Reasoning
  2. 2026LLMs as High-Dimensional Nonlinear Autoregressive Models with Attention: Training, Alignment and Inference
  3. 2023Beyond Words: A Mathematical Framework for Interpreting Large Language Models
  4. 2025Geometric Dynamics of Agentic Loops in Large Language Models
  5. 2024A critical review of methods and challenges in large language models
  6. 2026LLMs Explain't: A Post-Mortem on Semantic Interpretability in Transformer Models
  7. 2025Emergent Response Planning in LLMs
  8. 2023Guiding Large Language Models via Directional Stimulus Prompting