Riemann GeoResolver: A Non-Euclidean Attention Framework from Euclidean Resolver to Hyperbolic-Spherical Geometry
Not on the map. Why?
Abstract (arXiv)
We present a theoretical foundation for inverse-distance attention, from its Euclidean prototype (Resolver) to its non-Euclidean realization (Riemann GeoResolver). The Euclidean part establishes three core theorems: (1) circuit separation---IDA achieves exact retrieval with O(1) resources while softmax requires Omega((log n)2) width; (2) a Polyak--Lojasiewicz inequality with Omega(e^Delta2/√d/Delta2) stronger constant than softmax, implying linear convergence, O(log n) Lipschitz scaling under a low-rank/clustering assumption, Theta(1) Hessian spread, and absence of spurious local minima; (3) a width-independent effective rank bound that limits noise memorization---softmax memorizes arbitrary labels when dh≥ n, while IDA limits test error to O(η2). The non-Euclidean extension then builds upon this prototype, replacing Euclidean distance with hyperbolic geodesic distance for storage and spherical geodesic distance for routing. The Riemann GeoResolver framework comprises ten integrated modules: four HIDA operators spanning Theta(n2) to Theta(1) per token; Hyperbolic Curvature Compression (HCC) with provable error bounds; HyperGate with gradient lower-bound theorem; Spherical Inverse Distance Attention (SIDA) with sphere-analog PL inequalities; Dynamic Memory Genesis (DMG) with O(log T) regret bounds; and Geodesic Sparse Routing (GSR) with quality and communication bounds. The Euclidean theorems are proved in full; the non-Euclidean extension theorems are proved with analogous arguments. This work establishes a theoretical arc: from Euclidean attention as a special case, to hyperbolic memory, to spherical retrieval.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 51 of its 67 references that are in the map. First posted on arXiv 11 Aug 2026.
Builds on
51 of 67 references.
References · 51
- 2017Attention Is All You Needcited by 60,361
- 2016Layer Normalizationcited by 5,669
- 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
- 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
- 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
- 2020Longformer: The Long-Document Transformercited by 3,413
- 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
- 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
- 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
- 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 2,022
- 2016Identity Mappings in Deep Residual Networkscited by 1,965
- 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
38 more
- 2020Reformer: The Efficient Transformercited by 1,668
- 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,631
- 2019Generating Long Sequences with Sparse Transformerscited by 1,546
- 2020Linformer: Self-Attention with Linear Complexitycited by 1,474
- 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
- 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
- 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,336
- 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 927
- 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 897
- 2022Simplified State Space Layers for Sequence Modelingcited by 788
- 2018Image Transformercited by 772
- 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 717
- 2017Poincar'e Embeddings for Learning Hierarchical Representationscited by 684
- 2014Neural Turing Machinescited by 644
- 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 637
- 2016Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Conditioncited by 539
- 2018Spherical CNNscited by 456
- 2022On the Parameterization and Initialization of Diagonal State Space Modelscited by 424
- 2018Hyperbolic Neural Networkscited by 402
- 2019Hyperbolic Graph Convolutional Neural Networkscited by 402
- 2019Attention Augmented Convolutional Networkscited by 363
- 2022Diagonal State Spaces are as Effective as Structured State Spacescited by 350
- 2021BASE Layers: Simplifying Training of Large, Sparse Modelscited by 320
- 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 294
- 2018Learning Continuous Hierarchies in the Lorentz Model of Hyperbolic Geometrycited by 265
- 2017Learning SO(3) Equivariant Representations with Spherical CNNscited by 248
- 2021Hash Layers For Large Sparse Modelscited by 237
- 2016Dense Associative Memory for Pattern Recognitioncited by 236
- 2020Loss landscapes and optimization in over-parameterized non-linear systems and neural networkscited by 233
- 2018Representation Tradeoffs for Hyperbolic Embeddingscited by 206
- 2017Convergence Analysis of Two-layer Neural Networks with ReLU Activationcited by 137
- 2018Hyperbolic Attention Networkscited by 122
- 2021Fully Hyperbolic Neural Networkscited by 108
- 2022Hyperbolic Graph Neural Networks: A Review of Methods and Applicationscited by 40
- 2023The statistical thermodynamics of generative diffusion models: Phase transitions, symmetry breaking and critical instabilitycited by 39
- 2025Hyperbolic Deep Learning for Foundation Models: A Surveycited by 20
- 2026STAR-KV: Low-Rank KV Cache Compression via Soft Thresholding for Adaptive Rank Controlcited by 11
- 2025Attention on the Spherecited by 10
50 most cited of 51 All 51 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.
- 2026HCC+: Hyperbolic Guarding for Certified Attention Retrieval25 Aug 2026