# When Should Graph Attention Be Sparse? Learning a Per-Edge Tsallis Index arXiv: https://arxiv.org/abs/2608.02938 · HTML: https://mlsanity.com/paper/2608.02938/ ## Key facts - Cited by 1 papers in the map (data through 14 Oct 2026). - Builds on 35 of its 52 references that are in the map. - First posted on arXiv 3 Aug 2026. ## Builds on (most cited first, 30 of 35) - [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016) - [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017) - [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017) - [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015) - [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017) - [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018) - [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017) - [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019) - [Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels](https://mlsanity.com/paper/1805.07836/) (2018) - [When Does Label Smoothing Help?](https://mlsanity.com/paper/1906.02629/) (2019) - [Revisiting Semi-Supervised Learning with Graph Embeddings](https://mlsanity.com/paper/1603.08861/) (2016) - [Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks](https://mlsanity.com/paper/1810.02244/) (2018) - [How Attentive are Graph Attention Networks?](https://mlsanity.com/paper/2105.14491/) (2021) - [Geom-GCN: Geometric Graph Convolutional Networks](https://mlsanity.com/paper/2002.05287/) (2020) - [Recipe for a General, Powerful, Scalable Graph Transformer](https://mlsanity.com/paper/2205.12454/) (2022) - [Adaptive Universal Generalized PageRank Graph Neural Network](https://mlsanity.com/paper/2006.07988/) (2020) - [Robust Loss Functions under Label Noise for Deep Neural Networks](https://mlsanity.com/paper/1712.09482/) (2017) - [Beyond Low-frequency Information in Graph Convolutional Networks](https://mlsanity.com/paper/2101.00797/) (2021) - [Calibrating Deep Neural Networks using Focal Loss](https://mlsanity.com/paper/2002.09437/) (2020) - [A critical look at the evaluation of GNNs under heterophily: Are we really making progress?](https://mlsanity.com/paper/2302.11640/) (2023) - [From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classification](https://mlsanity.com/paper/1602.02068/) (2016) - [Do Transformers Really Perform Bad for Graph Representation?](https://mlsanity.com/paper/2106.05234/) (2021) - [Large Scale Learning on Non-Homophilous Graphs: New Benchmarks and Strong Simple Methods](https://mlsanity.com/paper/2110.14446/) (2021) - [On Mixup Training: Improved Calibration and Predictive Uncertainty for Deep Neural Networks](https://mlsanity.com/paper/1905.11001/) (2019) - [A General and Adaptive Robust Loss Function](https://mlsanity.com/paper/1701.03077/) (2017) - [Finding Global Homophily in Graph Neural Networks When Meeting Heterophily](https://mlsanity.com/paper/2205.07308/) (2022) - [Sparse Sequence-to-Sequence Models](https://mlsanity.com/paper/1905.05702/) (2019) - [Adaptively Sparse Transformers](https://mlsanity.com/paper/1909.00015/) (2019) - [Learning with Fenchel-Young Losses](https://mlsanity.com/paper/1901.02324/) (2019) - [Polynormer: Polynomial-Expressive Graph Transformer in Linear Time](https://mlsanity.com/paper/2403.01232/) (2024) ## Built on by (most cited first, 1 of 1) - [Generalized Graph Variational Autoencoders: Bounded Divergences Control Posterior Collapse](https://mlsanity.com/paper/2609.29546/) (2026) ## Abstract (arXiv) Graph attention normalizes neighborhood scores with softmax, the maximum-entropy choice under Shannon statistics. But homophilic and heterophilic graphs want different attention shapes, and one fixed normalization cannot serve both. We propose LTGA (Learnable Tsallis Graph Attention), a graph attention layer whose Tsallis entropic index q is learned jointly with the weights, interpolating continuously between heavy-tailed (q!!1) attention at four granularities from a global scalar to a per-edge index, under a bounded reparameterization that starts every model at the GAT baseline. Across eight benchmarks at ten seeds, LTGA-Edge takes the best average rank (2.75), but the omnibus test does not reject (p!=!0.199) and learning q does not beat searching it: a validation-tuned frozen grid reaches 61.4%, tuned α-entmax 62.2% and a capacity-matched q!equiv!1 control 62.0%, against 61.7% for LTGA-Edge. What the learned index buys is one run instead of a grid, and an interpretable mechanism: where q leaves 1, it prunes 42% of attention coefficients to exactly zero, and those edges are selectively the wrong ones, restoring them costs 7.1 points, while random pruning at the same rate costs 13.0 more. Project page: https://kleyt0n.github.io/ltga