# Amplitude-Only FFN Intervention for Tool-Structured LLM Inference: Method, Gated Evaluation Protocol, and Cross-Model Empirical Results

arXiv: https://arxiv.org/abs/2607.11183 · HTML: https://mlsanity.com/paper/2607.11183/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 24 references that are in the map.
- First posted on arXiv 13 Jul 2026.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://mlsanity.com/paper/2307.16789/) (2023)
- [MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark](https://mlsanity.com/paper/2406.01574/) (2024)
- [GLU Variants Improve Transformer](https://mlsanity.com/paper/2002.05202/) (2020)
- [Representation Engineering: A Top-Down Approach to AI Transparency](https://mlsanity.com/paper/2310.01405/) (2023)
- [DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence](https://mlsanity.com/paper/2606.19348/) (2026)
- [C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Models](https://mlsanity.com/paper/2305.08322/) (2023)
- [Muon is Scalable for LLM Training](https://mlsanity.com/paper/2502.16982/) (2025)
- [L-Eval: Instituting Standardized Evaluation for Long Context Language Models](https://mlsanity.com/paper/2307.11088/) (2023)
- [Tool Learning with Large Language Models: A Survey](https://mlsanity.com/paper/2405.17935/) (2024)
- [Efficient Guided Generation for Large Language Models](https://mlsanity.com/paper/2307.09702/) (2023)
- [ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answering](https://mlsanity.com/paper/2210.03849/) (2022)
- [Optimal strategies for reject option classifiers](https://mlsanity.com/paper/2101.12523/) (2021)
- [Fast Feedforward Networks](https://mlsanity.com/paper/2308.14711/) (2023)
- [Satisfiability Solving with LLMs: A Matched-Pair Evaluation of Reasoning Capability](https://mlsanity.com/paper/2605.28602/) (2026)

## Abstract (arXiv)

We study Amplitude Gating (AG), an inference-time intervention that masks feed-forward network (FFN) activation coordinates while keeping pretrained weights fixed. Six SwiGLU sites separate shared and gate-only inputs, expanded gate/value channels, and the FFN output. Magnitude-ranked entropy masks and a Newton-Schulz windowed ranking variant provide alternative forward paths without retraining. The corrected September collection covers three Qwen checkpoints and eight text/tool endpoints. Under frozen local text scorers, the displayed Qwen3.5-9B non-NS configurations improve two function-call endpoints by +2.61 and +2.67 percentage points (pp), and static ToolBench in/out-domain endpoints by +1.64 and +2.82 pp. Comparing the lower historical/September AG mean with the higher baseline mean leaves positive margins on these routes. This supports robustness to the two observed executions, rather than formal statistical significance. JSON gains are small and displayed L-Eval effects are negative. These endpoints measure lexical or answer/action-name agreement rather than executed-tool success. Protocol-specific offline selector and Per-Sample Fix-Harm Evaluation (PFHE) analyses complement the fixed-route results. Current experimental data do not establish a significant random-forest (RF) benefit; further experiments are needed. AG provides a task-dependent intervention surface, with causal mechanisms, equal-budget advantages, repeated-run uncertainty, and serving cost still open.