When Should Graph Attention Be Sparse? Learning a Per-Edge Tsallis Index
Not on the map. Why?
Abstract (arXiv)
Graph attention normalizes neighborhood scores with softmax, the maximum-entropy choice under Shannon statistics. But homophilic and heterophilic graphs want different attention shapes, and one fixed normalization cannot serve both. We propose LTGA (Learnable Tsallis Graph Attention), a graph attention layer whose Tsallis entropic index q is learned jointly with the weights, interpolating continuously between heavy-tailed (q!<!1), softmax (q!=!1) and compact-support (q!>!1) attention at four granularities from a global scalar to a per-edge index, under a bounded reparameterization that starts every model at the GAT baseline. Across eight benchmarks at ten seeds, LTGA-Edge takes the best average rank (2.75), but the omnibus test does not reject (p!=!0.199) and learning q does not beat searching it: a validation-tuned frozen grid reaches 61.4%, tuned α-entmax 62.2% and a capacity-matched q!equiv!1 control 62.0%, against 61.7% for LTGA-Edge. What the learned index buys is one run instead of a grid, and an interpretable mechanism: where q leaves 1, it prunes 42% of attention coefficients to exactly zero, and those edges are selectively the wrong ones, restoring them costs 7.1 points, while random pruning at the same rate costs 13.0 more. Project page: https://kleyt0n.github.io/ltga
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 35 of its 52 references that are in the map. First posted on arXiv 3 Aug 2026.
Builds on
35 of 52 references.
References · 35
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
- 2017Graph Attention Networkscited by 8,923
- 2017Inductive Representation Learning on Large Graphscited by 6,320
- 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
- 2017On Calibration of Modern Neural Networkscited by 4,960
- 2018How Powerful are Graph Neural Networks?cited by 4,393
- 2017Neural Message Passing for Quantum Chemistrycited by 3,487
- 2019A Comprehensive Survey on Graph Neural Networkscited by 2,919
- 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 997
- 2019When Does Label Smoothing Help?cited by 988
- 2016Revisiting Semi-Supervised Learning with Graph Embeddingscited by 976
- 2018Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networkscited by 881
23 more
- 2021How Attentive are Graph Attention Networks?cited by 835
- 2020Geom-GCN: Geometric Graph Convolutional Networkscited by 826
- 2022Recipe for a General, Powerful, Scalable Graph Transformercited by 713
- 2020Adaptive Universal Generalized PageRank Graph Neural Networkcited by 595
- 2017Robust Loss Functions under Label Noise for Deep Neural Networkscited by 443
- 2021Beyond Low-frequency Information in Graph Convolutional Networkscited by 362
- 2020Calibrating Deep Neural Networks using Focal Losscited by 357
- 2023A critical look at the evaluation of GNNs under heterophily: Are we really making progress?cited by 343
- 2016From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classificationcited by 341
- 2021Do Transformers Really Perform Bad for Graph Representation?cited by 320
- 2021Large Scale Learning on Non-Homophilous Graphs: New Benchmarks and Strong Simple Methodscited by 311
- 2019On Mixup Training: Improved Calibration and Predictive Uncertainty for Deep Neural Networkscited by 305
- 2017A General and Adaptive Robust Loss Functioncited by 236
- 2022Finding Global Homophily in Graph Neural Networks When Meeting Heterophilycited by 179
- 2019Sparse Sequence-to-Sequence Modelscited by 170
- 2019Adaptively Sparse Transformerscited by 155
- 2019Learning with Fenchel-Young Lossescited by 119
- 2024Polynormer: Polynomial-Expressive Graph Transformer in Linear Timecited by 65
- 2018SparseMAP: Differentiable Sparse Structured Inferencecited by 62
- 2019Robust Bi-Tempered Logistic Loss Based on Bregman Divergencescited by 61
- 2018On Controllable Sparse Alternatives to Softmaxcited by 27
- 2016Tsallis Regularized Optimal Transport and Ecological Inferencecited by 15
- 2020Evidential Sparsification of Multimodal Latent Spaces in Conditional Variational Autoencoderscited by 15
Built on by
Citing papers per year: 2026: 1.
1 paper.