# Reading the Gate, Not the Interference: Output-Side Interference Measurement Does Not Track Merge Collapse arXiv: https://arxiv.org/abs/2608.11797 ยท HTML: https://mlsanity.com/paper/2608.11797/ ## Key facts - Cited by 1 papers in the map (data through 14 Oct 2026). - Builds on 30 of its 35 references that are in the map. - First posted on arXiv 12 Aug 2026. ## Builds on (most cited first, 30 of 30) - [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021) - [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021) - [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024) - [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018) - [Editing Models with Task Arithmetic](https://mlsanity.com/paper/2212.04089/) (2022) - [TIES-Merging: Resolving Interference When Merging Models](https://mlsanity.com/paper/2306.01708/) (2023) - [Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch](https://mlsanity.com/paper/2311.03099/) (2023) - [PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preference](https://mlsanity.com/paper/2406.15513/) (2024) - [Task Arithmetic in the Tangent Space: Improved Editing of Pre-Trained Models](https://mlsanity.com/paper/2305.12827/) (2023) - [The Hydra Effect: Emergent Self-repair in Language Model Computations](https://mlsanity.com/paper/2307.15771/) (2023) - [Task Singular Vectors: Reducing Task Interference in Model Merging](https://mlsanity.com/paper/2412.00081/) (2024) - [Localizing Task Information for Improved Model Merging and Compression](https://mlsanity.com/paper/2405.07813/) (2024) - [Defeating the Training-Inference Mismatch via FP16](https://mlsanity.com/paper/2510.26788/) (2025) - [Whoever Started the Interference Should End It: Guiding Data-Free Model Merging via Task Vectors](https://mlsanity.com/paper/2503.08099/) (2025) - [Explorations of Self-Repair in Language Models](https://mlsanity.com/paper/2402.15390/) (2024) - [Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models](https://mlsanity.com/paper/2601.14004/) (2026) - [On the Emergence of Cross-Task Linearity in the Pretraining-Finetuning Paradigm](https://mlsanity.com/paper/2402.03660/) (2024) - [Towards Minimizing Feature Drift in Model Merging: Layer-wise Task Vector Fusion for Adaptive Knowledge Integration](https://mlsanity.com/paper/2505.23859/) (2025) - [SurgeryV2: Bridging the Gap Between Model Merging and Multi-Task Learning with Deep Representation Surgery](https://mlsanity.com/paper/2410.14389/) (2024) - [The Curse of Multiple Mediators: Hidden Interaction Effects in Activation Patching](https://mlsanity.com/paper/2606.27510/) (2026) - [RAIN-Merging: A Gradient-Free Method to Enhance Instruction Following in Large Reasoning Models with Preserved Thinking Format](https://mlsanity.com/paper/2602.22538/) (2026) - [DC-Merge: Improving Model Merging with Directional Consistency](https://mlsanity.com/paper/2603.06242/) (2026) - [Addressing divergent representations from causal interventions on neural networks](https://mlsanity.com/paper/2511.04638/) (2025) - [Demystifying Mergeability: Interpretable Properties to Predict Model Merging Success](https://mlsanity.com/paper/2601.22285/) (2026) - [Layer-Aware Task Arithmetic: Disentangling Task-Specific and Instruction-Following Knowledge](https://mlsanity.com/paper/2502.20186/) (2025) - [From Memorization to Parameter Interference: How Overtraining Experts Harms Model Merging](https://mlsanity.com/paper/2506.14126/) (2025) - [An Empirical Study and Theoretical Explanation on Task-Level Model-Merging Collapse](https://mlsanity.com/paper/2603.09463/) (2026) - [Activated Parameter Locating via Causal Intervention for Model Merging](https://mlsanity.com/paper/2408.09485/) (2024) - [Learning Not To Interfere: Adapting Experts For Model Merging](https://mlsanity.com/paper/2603.13467/) (2026) - [TwistedMerge: Certified Higher-Order Diagnostics and Abstention for Model Merging](https://mlsanity.com/paper/2607.20887/) (2026) ## Built on by (most cited first, 1 of 1) - [Published Unlearning Numbers Move Per Checkpoint, and Not Because the Removed Data Survives: An Audit of 263 Released Batch-Normalized Checkpoints](https://mlsanity.com/paper/2609.11490/) (2026) ## Abstract (arXiv) Task-arithmetic merging works until it doesn't, and the field diagnoses why by measuring interference inside the merged model. We take the most direct such measure, the exact layerwise activation cross-term of a factorial ledger, establish its causal anatomy, and then ask what it tracks. The anatomy is clean: each block mostly transports and amplifies the cross-term rather than generating it; erased, it is regenerated by the untouched marginal paths to 99% of its norm unless removed late; its output effect varies monotonically with the displacement's angle (orthogonal displacements make interference worse), and a two-assumption model derives the angle law and retro-dicts the dose curve (R^2 >= 0.99). What the measure tracks is not what the field assumes. Behavioural expert-likeness is decoupled from it across four instruments. Its cross-condition behaviour is denominator-dominated: an instruction template pins the main effect to within 1% while the absolute interaction grows 111x from two to six merged tasks, suppressing expressed interference at k=2 and amplifying it at k=6. And where merging actually collapses, the cross-term is a bystander, not the carrier: across two collapse parameterizations at two scales, even erased persistently at every position, removing it entirely repairs none of the collapse. There the output-side ratio carries no method information under a common counterfactual, while two state-space measures the field already uses rank methods correctly at both scales. All 81 predictions were frozen before their data; falsifications are reported as such. Output-side interference measurement reads the gate, the denominator, and the displacement budget, not the interference. What fails a merge is the carrier-bystander split: collapse rides in the marginal displacements while the cross-term merely accompanies it, and only state space sees the carrier.