Safin-1: Safety from Within through Memory-Native State Evolution
Not on the map. Why?
Abstract (arXiv)
Long-horizon complex tasks require foundation models to accumulate information, maintain internal states, and adapt over extended interactions. Safety should be an intrinsic property of the model itself, rather than a behavioral constraint relying solely on external safeguards or post-hoc alignment such as supervised fine-tuning. This motivates Safety from Within, where safety-relevant capabilities are represented and invoked through the model's native computation. We present Safin-1, a family of foundation models realizing this principle through memory routing and state evolution. Safin-1 is built on Memory-Anchor Routing across Context History (MARCH), a network architecture that maintains structured memory states and selectively retrieves relevant historical information through content-conditioned routing. It supports test-time adaptation of persistent capability states without repeatedly modifying the backbone, enabling controlled specialization over a shared foundation. We investigate this interface on downstream safety tasks through a Safety State, demonstrating effective state-based adaptation with substantial safety improvements. More broadly, the routed-state interface unifies contextual memory and persistent capability adaptation within the model's native computation, reframing memory from a passive record of prior context into an active substrate for maintaining and evolving model behavior. Evaluations across general capabilities, long-context understanding, retrieval, and efficiency further validate Safin-1. These findings provide a path toward safety as a state-native and adaptively maintainable capability. This work is only an initial architectural exploration of Safety from Within, and substantial further work is needed to realize this broader vision.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 108 of its 123 references that are in the map. First posted on arXiv 31 Aug 2026.
Builds on
108 of 123 references.
References · 108
- 2017Attention Is All You Needcited by 60,361
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2021Evaluating Large Language Models Trained on Codecited by 8,320
- 2020Measuring Massive Multitask Language Understandingcited by 7,963
- 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
- 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
- 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
- 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
- 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
- 2021Program Synthesis with Large Language Modelscited by 3,637
- 2020Longformer: The Long-Document Transformercited by 3,413
38 more
- 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
- 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
- 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
- 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
- 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
- 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,164
- 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 2,022
- 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
- 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
- 2020Big Bird: Transformers for Longer Sequencescited by 1,684
- 2020Rethinking Attention with Performerscited by 1,601
- 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
- 2019Generating Long Sequences with Sparse Transformerscited by 1,546
- 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
- 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
- 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
- 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,336
- 2023Instruction-Following Evaluation for Large Language Modelscited by 1,228
- 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
- 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
- 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 992
- 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 903
- 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 832
- 2023RWKV: Reinventing RNNs for the Transformer Eracited by 827
- 2023Retentive Network: A Successor to Transformer for Large Language Modelscited by 613
- 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 536
- 2024Gated Delta Networks: Improving Mamba2 with Delta Rulecited by 532
- 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 505
- 2021Linear Transformers Are Secretly Fast Weight Programmerscited by 502
- 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inferencecited by 501
- 2020Efficient Content-Based Sparse Attention with Routing Transformerscited by 471
- 2021Nystr"omformer: A Nystr"om-Based Algorithm for Approximating Self-Attentioncited by 471
- 2023Resurrecting Recurrent Neural Networks for Long Sequencescited by 408
- 2025Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attentioncited by 407
- 2024Parallelizing Linear Transformers with the Delta Rule over Sequence Lengthcited by 396
- 2023Learning to Compress Prompts with Gist Tokenscited by 390
- 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 387
- 2024A StrongREJECT for Empty Jailbreakscited by 385
50 most cited of 108 All 108 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.