Graph Laplacian Wavelet Transformer via Learnable Spectral Decomposition
Not on the map. Why?
Abstract (arXiv)
Existing sequence to sequence models for structured language tasks rely heavily on the dot product self attention mechanism, which incurs quadratic complexity in both computation and memory for input length N. We introduce the Graph Wavelet Transformer (GWT), a novel architecture that replaces this bottleneck with a learnable, multi scale wavelet transform defined over an explicit graph Laplacian derived from syntactic or semantic parses. Our analysis shows that multi scale spectral decomposition offers an interpretable, efficient, and expressive alternative to quadratic self attention for graph structured sequence modeling.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 20 of its 37 references that are in the map. First posted on arXiv 9 May 2025.
Builds on
20 of 37 references.
References · 20
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
- 2017Graph Attention Networkscited by 8,923
- 2020Longformer: The Long-Document Transformercited by 3,413
- 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
- 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 2,022
- 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
- 2020Reformer: The Efficient Transformercited by 1,668
- 2020Rethinking Attention with Performerscited by 1,601
- 2019Generating Long Sequences with Sparse Transformerscited by 1,546
8 more
- 2020Linformer: Self-Attention with Linear Complexitycited by 1,474
- 2013Spectral Networks and Locally Connected Networks on Graphscited by 1,102
- 2020Efficient Transformers: A Surveycited by 790
- 2020A Generalization of Transformer Networks to Graphscited by 661
- 2021Nystr"omformer: A Nystr"om-Based Algorithm for Approximating Self-Attentioncited by 471
- 2017Regularizing and Optimizing LSTM Language Modelscited by 205
- 2017Graph Convolutional Encoders for Syntax-aware Neural Machine Translationcited by 70
- 2025Learnable Multi-Scale Wavelet Transformer: A Novel Alternative to Self-Attentioncited by 7
Built on by
Citing papers per year: 2025: 1.
1 paper.