mlsanityData to 14 Oct 2026

Safin-1: Safety from Within through Memory-Native State Evolution

arXiv 2609.00092 ↗First posted 31 Aug 2026cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Long-horizon complex tasks require foundation models to accumulate information, maintain internal states, and adapt over extended interactions. Safety should be an intrinsic property of the model itself, rather than a behavioral constraint relying solely on external safeguards or post-hoc alignment such as supervised fine-tuning. This motivates Safety from Within, where safety-relevant capabilities are represented and invoked through the model's native computation. We present Safin-1, a family of foundation models realizing this principle through memory routing and state evolution. Safin-1 is built on Memory-Anchor Routing across Context History (MARCH), a network architecture that maintains structured memory states and selectively retrieves relevant historical information through content-conditioned routing. It supports test-time adaptation of persistent capability states without repeatedly modifying the backbone, enabling controlled specialization over a shared foundation. We investigate this interface on downstream safety tasks through a Safety State, demonstrating effective state-based adaptation with substantial safety improvements. More broadly, the routed-state interface unifies contextual memory and persistent capability adaptation within the model's native computation, reframing memory from a passive record of prior context into an active substrate for maintaining and evolving model behavior. Evaluations across general capabilities, long-context understanding, retrieval, and efficiency further validate Safin-1. These findings provide a path toward safety as a state-native and adaptively maintainable capability. This work is only an initial architectural exploration of Safety from Within, and substantial further work is needed to realize this broader vision.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 108 of its 123 references that are in the map. First posted on arXiv 31 Aug 2026.

Builds on

108 of 123 references.

References · 108

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  7. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  8. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  9. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  10. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  11. 2021Program Synthesis with Large Language Modelscited by 3,637
  12. 2020Longformer: The Long-Document Transformercited by 3,413
38 more
  1. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  2. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  3. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  4. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  5. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
  6. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,164
  7. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 2,022
  8. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  9. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
  10. 2020Big Bird: Transformers for Longer Sequencescited by 1,684
  11. 2020Rethinking Attention with Performerscited by 1,601
  12. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
  13. 2019Generating Long Sequences with Sparse Transformerscited by 1,546
  14. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  15. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
  16. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  17. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,336
  18. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,228
  19. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  20. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
  21. 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 992
  22. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 903
  23. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 832
  24. 2023RWKV: Reinventing RNNs for the Transformer Eracited by 827
  25. 2023Retentive Network: A Successor to Transformer for Large Language Modelscited by 613
  26. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 536
  27. 2024Gated Delta Networks: Improving Mamba2 with Delta Rulecited by 532
  28. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 505
  29. 2021Linear Transformers Are Secretly Fast Weight Programmerscited by 502
  30. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inferencecited by 501
  31. 2020Efficient Content-Based Sparse Attention with Routing Transformerscited by 471
  32. 2021Nystr"omformer: A Nystr"om-Based Algorithm for Approximating Self-Attentioncited by 471
  33. 2023Resurrecting Recurrent Neural Networks for Long Sequencescited by 408
  34. 2025Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attentioncited by 407
  35. 2024Parallelizing Linear Transformers with the Delta Rule over Sequence Lengthcited by 396
  36. 2023Learning to Compress Prompts with Gist Tokenscited by 390
  37. 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 387
  38. 2024A StrongREJECT for Empty Jailbreakscited by 385

50 most cited of 108 All 108 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.