mlsanityData to 14 Oct 2026

When Should Graph Attention Be Sparse? Learning a Per-Edge Tsallis Index

arXiv 2608.02938 ↗First posted 3 Aug 2026cs.LG cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Graph attention normalizes neighborhood scores with softmax, the maximum-entropy choice under Shannon statistics. But homophilic and heterophilic graphs want different attention shapes, and one fixed normalization cannot serve both. We propose LTGA (Learnable Tsallis Graph Attention), a graph attention layer whose Tsallis entropic index q is learned jointly with the weights, interpolating continuously between heavy-tailed (q!<!1), softmax (q!=!1) and compact-support (q!>!1) attention at four granularities from a global scalar to a per-edge index, under a bounded reparameterization that starts every model at the GAT baseline. Across eight benchmarks at ten seeds, LTGA-Edge takes the best average rank (2.75), but the omnibus test does not reject (p!=!0.199) and learning q does not beat searching it: a validation-tuned frozen grid reaches 61.4%, tuned α-entmax 62.2% and a capacity-matched q!equiv!1 control 62.0%, against 61.7% for LTGA-Edge. What the learned index buys is one run instead of a grid, and an interpretable mechanism: where q leaves 1, it prunes 42% of attention coefficients to exactly zero, and those edges are selectively the wrong ones, restoring them costs 7.1 points, while random pruning at the same rate costs 13.0 more. Project page: https://kleyt0n.github.io/ltga

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 35 of its 52 references that are in the map. First posted on arXiv 3 Aug 2026.

Builds on

35 of 52 references.

References · 35

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  2. 2017Graph Attention Networkscited by 8,923
  3. 2017Inductive Representation Learning on Large Graphscited by 6,320
  4. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  5. 2017On Calibration of Modern Neural Networkscited by 4,960
  6. 2018How Powerful are Graph Neural Networks?cited by 4,393
  7. 2017Neural Message Passing for Quantum Chemistrycited by 3,487
  8. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,919
  9. 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 997
  10. 2019When Does Label Smoothing Help?cited by 988
  11. 2016Revisiting Semi-Supervised Learning with Graph Embeddingscited by 976
  12. 2018Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networkscited by 881
23 more
  1. 2021How Attentive are Graph Attention Networks?cited by 835
  2. 2020Geom-GCN: Geometric Graph Convolutional Networkscited by 826
  3. 2022Recipe for a General, Powerful, Scalable Graph Transformercited by 713
  4. 2020Adaptive Universal Generalized PageRank Graph Neural Networkcited by 595
  5. 2017Robust Loss Functions under Label Noise for Deep Neural Networkscited by 443
  6. 2021Beyond Low-frequency Information in Graph Convolutional Networkscited by 362
  7. 2020Calibrating Deep Neural Networks using Focal Losscited by 357
  8. 2023A critical look at the evaluation of GNNs under heterophily: Are we really making progress?cited by 343
  9. 2016From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classificationcited by 341
  10. 2021Do Transformers Really Perform Bad for Graph Representation?cited by 320
  11. 2021Large Scale Learning on Non-Homophilous Graphs: New Benchmarks and Strong Simple Methodscited by 311
  12. 2019On Mixup Training: Improved Calibration and Predictive Uncertainty for Deep Neural Networkscited by 305
  13. 2017A General and Adaptive Robust Loss Functioncited by 236
  14. 2022Finding Global Homophily in Graph Neural Networks When Meeting Heterophilycited by 179
  15. 2019Sparse Sequence-to-Sequence Modelscited by 170
  16. 2019Adaptively Sparse Transformerscited by 155
  17. 2019Learning with Fenchel-Young Lossescited by 119
  18. 2024Polynormer: Polynomial-Expressive Graph Transformer in Linear Timecited by 65
  19. 2018SparseMAP: Differentiable Sparse Structured Inferencecited by 62
  20. 2019Robust Bi-Tempered Logistic Loss Based on Bregman Divergencescited by 61
  21. 2018On Controllable Sparse Alternatives to Softmaxcited by 27
  22. 2016Tsallis Regularized Optimal Transport and Ecological Inferencecited by 15
  23. 2020Evidential Sparsification of Multimodal Latent Spaces in Conditional Variational Autoencoderscited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.