mlsanityData to 14 Oct 2026

Safe Context Switching for Agents in the Wild: Mitigating Subspace Interference via Orthogonal Adaptation

arXiv 2610.05219 ↗First posted 4 Oct 2026cs.AI cs.CLcited by 0

On the map: LLM reasoning → Reasoning for agents, planning and code also fits: Test-time compute and efficient thinking

also in 3 more
Abstract (arXiv)

Most Large Language Models exhibit a fundamental tension between two sequential tasks, such as logical reasoning and safety alignment. The high-variance internal states required for sophisticated Chain-of-Thought (CoT) deduction can geometrically interfere with latent representations encoding safety constraints. We identify this phenomenon as Sequential Subspace Interference, showing that standard fine-tuning on logical tasks such as multi-step mathematics and code generation can result in a 23.3% interference penalty on alignment benchmarks, substantially weakening the model's safety priors. This Reasoning Drift is not adequately captured by current adaptation methods because gradients for logical tasks are rarely orthogonal to safety objectives. To address this issue, we propose AURA (Adaptive Unique Residual Allocation), a spectral regularization framework that enforces Spectral Independence between reasoning and safety. By explicitly estimating the null space of the alignment manifold and constraining reasoning updates to its orthogonal complement, AURA enables models to improve logical reasoning without compromising safety. Empirically, AURA recovers 23.0% of the lost performance while preserving greater than 0.98 cosine fidelity to the safe state, demonstrating that reasoning and alignment can be effectively decoupled through geometric regularization.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 11 of its 13 references that are in the map. First posted on arXiv 4 Oct 2026.

Builds on

11 of 13 references.

References · 11

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  3. 2021Program Synthesis with Large Language Modelscited by 3,637
  4. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  5. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,228
  6. 2022Editing Models with Task Arithmeticcited by 1,120
  7. 2018Efficient Lifelong Learning with A-GEMcited by 1,058
  8. 2020AdapterFusion: Non-Destructive Task Composition for Transfer Learningcited by 788
  9. 2023Orthogonal Subspace Learning for Language Model Continual Learningcited by 312
  10. 2025StelLA: Subspace Learning in Low-rank Adaptation using Stiefel Manifoldcited by 14
  11. 2025Activated LoRA: Fine-tuned LLMs for Intrinsicscited by 12

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Spectral Rewiring for Exploration, Purification, and Model Merging
  2. 2025A Guardrail for Safety Preservation: When Safety-Sensitive Subspace Meets Harmful-Resistant Null-Space
  3. 2026SafeAnchor: Preventing Cumulative Safety Erosion in Continual Domain Adaptation of Large Language Models
  4. 2026Safety Anchor: Defending Harmful Fine-tuning via Geometric Bottlenecks
  5. 2025Beyond SFT: Reinforcement Learning for Safer Large Reasoning Models with Better Reasoning Ability
  6. 2026Knowing without Acting: The Disentangled Geometry of Safety Mechanisms in Large Language Models
  7. 2025Decoupling Safety into Orthogonal Subspace: Cost-Efficient and Performance-Preserving Alignment for Large Language Models
  8. 2025EcoAlign: An Economically Rational Framework for Efficient LVLM Alignment