mlsanityData to 14 Oct 2026

Reading the Gate, Not the Interference: Output-Side Interference Measurement Does Not Track Merge Collapse

arXiv 2608.11797 ↗First posted 12 Aug 2026cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Task-arithmetic merging works until it doesn't, and the field diagnoses why by measuring interference inside the merged model. We take the most direct such measure, the exact layerwise activation cross-term of a factorial ledger, establish its causal anatomy, and then ask what it tracks. The anatomy is clean: each block mostly transports and amplifies the cross-term rather than generating it; erased, it is regenerated by the untouched marginal paths to 99% of its norm unless removed late; its output effect varies monotonically with the displacement's angle (orthogonal displacements make interference worse), and a two-assumption model derives the angle law and retro-dicts the dose curve (R^2 >= 0.99). What the measure tracks is not what the field assumes. Behavioural expert-likeness is decoupled from it across four instruments. Its cross-condition behaviour is denominator-dominated: an instruction template pins the main effect to within 1% while the absolute interaction grows 111x from two to six merged tasks, suppressing expressed interference at k=2 and amplifying it at k=6. And where merging actually collapses, the cross-term is a bystander, not the carrier: across two collapse parameterizations at two scales, even erased persistently at every position, removing it entirely repairs none of the collapse. There the output-side ratio carries no method information under a common counterfactual, while two state-space measures the field already uses rank methods correctly at both scales. All 81 predictions were frozen before their data; falsifications are reported as such. Output-side interference measurement reads the gate, the denominator, and the displacement budget, not the interference. What fails a merge is the carrier-bystander split: collapse rides in the marginal displacements while the cross-term merely accompanies it, and only state space sees the carrier.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 30 of its 35 references that are in the map. First posted on arXiv 12 Aug 2026.

Builds on

30 of 35 references.

References · 30

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2024Qwen2.5 Technical Reportcited by 4,942
  4. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,317
  5. 2022Editing Models with Task Arithmeticcited by 1,120
  6. 2023TIES-Merging: Resolving Interference When Merging Modelscited by 818
  7. 2023Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunchcited by 694
  8. 2024PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preferencecited by 261
  9. 2023Task Arithmetic in the Tangent Space: Improved Editing of Pre-Trained Modelscited by 221
  10. 2023The Hydra Effect: Emergent Self-repair in Language Model Computationscited by 156
  11. 2024Task Singular Vectors: Reducing Task Interference in Model Mergingcited by 147
  12. 2024Localizing Task Information for Improved Model Merging and Compressioncited by 131
18 more
  1. 2025Defeating the Training-Inference Mismatch via FP16cited by 66
  2. 2025Whoever Started the Interference Should End It: Guiding Data-Free Model Merging via Task Vectorscited by 62
  3. 2024Explorations of Self-Repair in Language Modelscited by 39
  4. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 27
  5. 2024On the Emergence of Cross-Task Linearity in the Pretraining-Finetuning Paradigmcited by 26
  6. 2025Towards Minimizing Feature Drift in Model Merging: Layer-wise Task Vector Fusion for Adaptive Knowledge Integrationcited by 13
  7. 2024SurgeryV2: Bridging the Gap Between Model Merging and Multi-Task Learning with Deep Representation Surgerycited by 12
  8. 2026The Curse of Multiple Mediators: Hidden Interaction Effects in Activation Patchingcited by 11
  9. 2026RAIN-Merging: A Gradient-Free Method to Enhance Instruction Following in Large Reasoning Models with Preserved Thinking Formatcited by 9
  10. 2026DC-Merge: Improving Model Merging with Directional Consistencycited by 8
  11. 2025Addressing divergent representations from causal interventions on neural networkscited by 7
  12. 2026Demystifying Mergeability: Interpretable Properties to Predict Model Merging Successcited by 7
  13. 2025Layer-Aware Task Arithmetic: Disentangling Task-Specific and Instruction-Following Knowledgecited by 6
  14. 2025From Memorization to Parameter Interference: How Overtraining Experts Harms Model Mergingcited by 4
  15. 2026An Empirical Study and Theoretical Explanation on Task-Level Model-Merging Collapsecited by 4
  16. 2024Activated Parameter Locating via Causal Intervention for Model Mergingcited by 3
  17. 2026Learning Not To Interfere: Adapting Experts For Model Mergingcited by 1
  18. 2026TwistedMerge: Certified Higher-Order Diagnostics and Abstention for Model Mergingcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.