mlsanityData to 7 Oct 2026

Warranted Attention: Learning What to Pass from Attention to Prediction

arXiv 2606.30139 ↗First posted 29 Jun 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Relevance of information read by attention does not guarantee that its contribution benefits the current prediction. We propose Warrant, which learns how strongly attention-derived item contributions should be transmitted under the current query. Warrant applies learned item-wise permission before aggregation without renormalization, jointly controlling relative allocation and total transmission mass. Across backbones, we construct interfaces connecting these con- tributions to prediction scores or states and compare against ungated models on the same paths. On three CyGNet datasets, ungated paths reduce MRR, whereas learned permission partially or almost fully recovers the losses. In a 5-seed HotpotQA/RoBERTa experiment, distractors receive lower permission than gold support, while mean Support MRR rises from .9111 to .9141 and the unsupported selection rate falls from .2271 to .2215. Performance comparisons and contribution- level interventions across five task families reveal both the effects and limits of selective control. The results support learning the strength of contributions transmitted to prediction separately from attention relevance

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 43 of its 55 references that are in the map. First posted on arXiv 29 Jun 2026.

Builds on

43 of 55 references.

References · 43

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2017On Calibration of Modern Neural Networkscited by 4,860
  5. 2017Squeeze-and-Excitation Networkscited by 4,763
  6. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  7. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  8. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  9. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  10. 2020Longformer: The Long-Document Transformercited by 3,362
  11. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  12. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
31 more
  1. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  2. 2019Attention is not Explanationcited by 916
  3. 2017Selective Classification for Deep Neural Networkscited by 799
  4. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
  5. 2016Language Modeling with Gated Convolutional Networkscited by 688
  6. 2019Attention is not not Explanationcited by 629
  7. 2017Deep Interest Network for Click-Through Rate Predictioncited by 541
  8. 2020Inductive Representation Learning on Temporal Graphscited by 454
  9. 2019Predicting Dynamic Embedding Trajectory in Temporal Interaction Networkscited by 443
  10. 2019Is Attention Interpretable?cited by 387
  11. 2022Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classificationcited by 355
  12. 2017Residual Gated Graph ConvNetscited by 341
  13. 2025Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Freecited by 266
  14. 2015Highway Networkscited by 257
  15. 2022Mega: Moving Average Equipped Gated Attentioncited by 178
  16. 2023Towards Better Dynamic Graph Learning: New Architecture and Unified Librarycited by 154
  17. 2018Learning Sequence Encoders for Temporal Knowledge Graph Completioncited by 145
  18. 2023Do We Really Need Complicated Model Architectures For Temporal Networks?cited by 141
  19. 2021Global Self-Attention as a Replacement for Graph Convolutioncited by 113
  20. 2020Neural Spatio-Temporal Point Processescited by 92
  21. 2021Transformer Embeddings of Irregularly Spaced Events and Their Participantscited by 90
  22. 2020Learning from History: Modeling Temporal Knowledge Graphs with Sequential Copy-Generation Networkscited by 84
  23. 2016Improving Neural Language Models with a Continuous Cachecited by 81
  24. 2024Theory, Analysis, and Best Practices for Sigmoid Self-Attentioncited by 62
  25. 2021Adaptive Nearest Neighbor Machine Translationcited by 58
  26. 2023Quantifying Context Mixing in Transformerscited by 43
  27. 2022Temporal Knowledge Graph Reasoning with Historical Contrastive Learningcited by 42
  28. 2021Evidentiality-guided Generation for Knowledge-Intensive NLP Taskscited by 40
  29. 2016Gated-Attention Readers for Text Comprehensioncited by 37
  30. 2022Towards Robust k-Nearest-Neighbor Machine Translationcited by 15
  31. 2020xERTE: Explainable Reasoning on Temporal Knowledge Graphs for Forecasting Future Linkscited by 13

Built on by

Not cited yet.