mlsanityData to 14 Oct 2026

Riemann GeoResolver: A Non-Euclidean Attention Framework from Euclidean Resolver to Hyperbolic-Spherical Geometry

arXiv 2608.10416 ↗First posted 11 Aug 2026cs.DS cs.AI cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

We present a theoretical foundation for inverse-distance attention, from its Euclidean prototype (Resolver) to its non-Euclidean realization (Riemann GeoResolver). The Euclidean part establishes three core theorems: (1) circuit separation---IDA achieves exact retrieval with O(1) resources while softmax requires Omega((log n)2) width; (2) a Polyak--Lojasiewicz inequality with Omega(e^Delta2/√d/Delta2) stronger constant than softmax, implying linear convergence, O(log n) Lipschitz scaling under a low-rank/clustering assumption, Theta(1) Hessian spread, and absence of spurious local minima; (3) a width-independent effective rank bound that limits noise memorization---softmax memorizes arbitrary labels when dh≥ n, while IDA limits test error to O(η2). The non-Euclidean extension then builds upon this prototype, replacing Euclidean distance with hyperbolic geodesic distance for storage and spherical geodesic distance for routing. The Riemann GeoResolver framework comprises ten integrated modules: four HIDA operators spanning Theta(n2) to Theta(1) per token; Hyperbolic Curvature Compression (HCC) with provable error bounds; HyperGate with gradient lower-bound theorem; Spherical Inverse Distance Attention (SIDA) with sphere-analog PL inequalities; Dynamic Memory Genesis (DMG) with O(log T) regret bounds; and Geodesic Sparse Routing (GSR) with quality and communication bounds. The Euclidean theorems are proved in full; the non-Euclidean extension theorems are proved with analogous arguments. This work establishes a theoretical arc: from Euclidean attention as a special case, to hyperbolic memory, to spherical retrieval.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 51 of its 67 references that are in the map. First posted on arXiv 11 Aug 2026.

Builds on

51 of 67 references.

References · 51

  1. 2017Attention Is All You Needcited by 60,361
  2. 2016Layer Normalizationcited by 5,669
  3. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  4. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  5. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  6. 2020Longformer: The Long-Document Transformercited by 3,413
  7. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  8. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  9. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
  10. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 2,022
  11. 2016Identity Mappings in Deep Residual Networkscited by 1,965
  12. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
38 more
  1. 2020Reformer: The Efficient Transformercited by 1,668
  2. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,631
  3. 2019Generating Long Sequences with Sparse Transformerscited by 1,546
  4. 2020Linformer: Self-Attention with Linear Complexitycited by 1,474
  5. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  6. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  7. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,336
  8. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 927
  9. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 897
  10. 2022Simplified State Space Layers for Sequence Modelingcited by 788
  11. 2018Image Transformercited by 772
  12. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 717
  13. 2017Poincar'e Embeddings for Learning Hierarchical Representationscited by 684
  14. 2014Neural Turing Machinescited by 644
  15. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 637
  16. 2016Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Conditioncited by 539
  17. 2018Spherical CNNscited by 456
  18. 2022On the Parameterization and Initialization of Diagonal State Space Modelscited by 424
  19. 2018Hyperbolic Neural Networkscited by 402
  20. 2019Hyperbolic Graph Convolutional Neural Networkscited by 402
  21. 2019Attention Augmented Convolutional Networkscited by 363
  22. 2022Diagonal State Spaces are as Effective as Structured State Spacescited by 350
  23. 2021BASE Layers: Simplifying Training of Large, Sparse Modelscited by 320
  24. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 294
  25. 2018Learning Continuous Hierarchies in the Lorentz Model of Hyperbolic Geometrycited by 265
  26. 2017Learning SO(3) Equivariant Representations with Spherical CNNscited by 248
  27. 2021Hash Layers For Large Sparse Modelscited by 237
  28. 2016Dense Associative Memory for Pattern Recognitioncited by 236
  29. 2020Loss landscapes and optimization in over-parameterized non-linear systems and neural networkscited by 233
  30. 2018Representation Tradeoffs for Hyperbolic Embeddingscited by 206
  31. 2017Convergence Analysis of Two-layer Neural Networks with ReLU Activationcited by 137
  32. 2018Hyperbolic Attention Networkscited by 122
  33. 2021Fully Hyperbolic Neural Networkscited by 108
  34. 2022Hyperbolic Graph Neural Networks: A Review of Methods and Applicationscited by 40
  35. 2023The statistical thermodynamics of generative diffusion models: Phase transitions, symmetry breaking and critical instabilitycited by 39
  36. 2025Hyperbolic Deep Learning for Foundation Models: A Surveycited by 20
  37. 2026STAR-KV: Low-Rank KV Cache Compression via Soft Thresholding for Adaptive Rank Controlcited by 11
  38. 2025Attention on the Spherecited by 10

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.