mlsanityData to 14 Oct 2026

Amplitude-Only FFN Intervention for Tool-Structured LLM Inference: Method, Gated Evaluation Protocol, and Cross-Model Empirical Results

arXiv 2607.11183 ↗First posted 13 Jul 2026cs.CL cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We study Amplitude Gating (AG), an inference-time intervention that masks feed-forward network (FFN) activation coordinates while keeping pretrained weights fixed. Six SwiGLU sites separate shared and gate-only inputs, expanded gate/value channels, and the FFN output. Magnitude-ranked entropy masks and a Newton-Schulz windowed ranking variant provide alternative forward paths without retraining. The corrected September collection covers three Qwen checkpoints and eight text/tool endpoints. Under frozen local text scorers, the displayed Qwen3.5-9B non-NS configurations improve two function-call endpoints by +2.61 and +2.67 percentage points (pp), and static ToolBench in/out-domain endpoints by +1.64 and +2.82 pp. Comparing the lower historical/September AG mean with the higher baseline mean leaves positive margins on these routes. This supports robustness to the two observed executions, rather than formal statistical significance. JSON gains are small and displayed L-Eval effects are negative. These endpoints measure lexical or answer/action-name agreement rather than executed-tool success. Protocol-specific offline selector and Per-Sample Fix-Harm Evaluation (PFHE) analyses complement the fixed-route results. Current experimental data do not establish a significant random-forest (RF) benefit; further experiments are needed. AG provides a task-dependent intervention surface, with causal mechanisms, equal-budget advantages, repeated-run uncertainty, and serving cost still open.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 17 of its 24 references that are in the map. First posted on arXiv 13 Jul 2026.

Builds on

17 of 24 references.

References · 17

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  5. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  6. 2020GLU Variants Improve Transformercited by 1,804
  7. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  8. 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 1,040
  9. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 708
  10. 2025Muon is Scalable for LLM Trainingcited by 428
  11. 2023L-Eval: Instituting Standardized Evaluation for Long Context Language Modelscited by 233
  12. 2024Tool Learning with Large Language Models: A Surveycited by 223
5 more
  1. 2023Efficient Guided Generation for Large Language Modelscited by 200
  2. 2022ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answeringcited by 196
  3. 2021Optimal strategies for reject option classifierscited by 61
  4. 2023Fast Feedforward Networkscited by 11
  5. 2026Satisfiability Solving with LLMs: A Matched-Pair Evaluation of Reasoning Capabilitycited by 2

Built on by

Not cited yet.