mlsanityData to 14 Oct 2026

Cryptanalytic Extraction of Isolated Bias-Free GLU Feed-Forward Blocks by Antipodal Separation

arXiv 2608.06631 ↗First posted 6 Aug 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Cryptanalytic extraction has been demonstrated for ReLU networks, for networks using componentwise activations such as GELU or SiLU, and for a Transformer's final projection matrix. These methods do not recover the bias-free Gated Linear Unit (GLU) feed-forward blocks used in many modern language models. Such a block multiplies an activated linear projection by a second learned linear projection within each hidden unit, a two-branch structure absent from the network classes and final-layer setting addressed by those methods. We give a constructive, multi-stage forward-query recovery primitive for isolated bias-free GLU blocks. Finite-difference curvature supplies gate-direction candidates, and paired observations at x and -x separate gate magnitude, orientation, and value-branch coupling. Across high-precision targets, six Qwen layers, an 8,192-unit Llama subproblem, and a full-dimensional Gemma block all reach sub-percent median validation error. Four finite-precision configurations remain below 5 percent median error, but none reproduces every stored weight. These isolated-block experiments are not an end-to-end model-API attack: deriving the required internal block responses from final model outputs remains unsolved.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 6 of its 16 references that are in the map. First posted on arXiv 6 Aug 2026.

Builds on

6 of 16 references.

References · 6

  1. 2020GLU Variants Improve Transformercited by 1,804
  2. 2016Language Modeling with Gated Convolutional Networkscited by 702
  3. 2016Stealing Machine Learning Models via Prediction APIscited by 619
  4. 2024Stealing Part of a Production Language Modelcited by 159
  5. 2020Cryptanalytic Extraction of Neural Network Modelscited by 68
  6. 2024Beyond Slow Signs in High-fidelity Model Extractioncited by 7

Built on by

Not cited yet.