mlsanityData to 14 Oct 2026

After the Euclidean Highway: Hyperbolic Expert AI as the Next Innovation

arXiv 2607.17513 ↗First posted 20 Jul 2026cs.LG cs.AI cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Expert domains are trees; the Euclidean transformer is not, diluting parent-child structure exponentially at depth. The hyperbolic turn left one question unasked: not how much of a network to curve, but where curvature may touch the gradient. Placement is a law, not a knob: the same geometry on a trainable adapter collapses training (seventeen training collapses, ~220 GPU-hours), yet at the loss layer alone it trains without one -- this is HySAT (Hyperbolic Structure-Aware Training), hyperbolic losses at the loss layer only. Across six expert SLMs we constructed and deployed (Llama 3.1 and EXAONE 3.5; four adapter strategies; 18.0M-sample corpus; zero NaN over ~317K optimizer steps), a matched four-arm ablation isolates the preserved manifold invariant, and three propositions and a lemma prove why loss-only placement is stable where adapter-on-manifold is not. Four models are operationally deployed (one live, consumer-facing), two open-weight, with per-step traces and a seventeen-incident failure ledger on Zenodo (CC-BY-4.0).

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 49 references that are in the map. First posted on arXiv 20 Jul 2026.

Builds on

15 of 49 references.

References · 15

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  3. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  4. 2024LoRA Learns Less and Forgets Lesscited by 328
  5. 2017Don't Decay the Learning Rate, Increase the Batch Sizecited by 320
  6. 2024Universal Neurons in GPT2 Language Modelscited by 102
  7. 2021Learning in High Dimension Always Amounts to Extrapolationcited by 66
  8. 2024EXAONE 3.5: Series of Large Language Models for Real-world Use Casescited by 53
  9. 2022The Numerical Stability of Hyperbolic Representation Learningcited by 47
  10. 2024Hyperbolic Fine-Tuning for Large Language Modelscited by 27
  11. 2025HELM: Hyperbolic Large Language Models via Mixture-of-Curvature Expertscited by 26
  12. 2025Hyperbolic Large Language Modelscited by 6
3 more
  1. 2025Continual Hyperbolic Learning of Instances and Classescited by 5
  2. 2025HyperET: Efficient Training in Hyperbolic Space for Multi-modal Large Language Modelscited by 5
  3. 2025Understanding and Improving Hyperbolic Deep Reinforcement Learningcited by 3

Built on by

Not cited yet.