mlsanityData to 14 Oct 2026

Language Modeling with Hyperspherical Flows

arXiv 2605.11125 ↗First posted 11 May 2026cs.LGcited by 22

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Discrete Diffusion Language Models progressed rapidly as an alternative to autoregressive (AR) models, motivated by their parallel generation abilities. However, for tractability, discrete diffusion models sample from a factorized distribution, which is less expressive than AR. Recent Flow Language Models (FLMs) apply continuous flows to language, transporting noise to data with a deterministic ODE that avoids factorized sampling. FLMs operate on one-hot vectors whose dimension scales with the vocabulary size, making FLMs costly to train. Moreover, since all distinct one-hot embeddings are equidistant in ℓ2, adding Gaussian noise does not have a clear semantic interpretation (unlike images, where Gaussian noise progressively degrades structure). We introduce mathbbS-FLM, a latent FLM in the hypersphere. mathbbS-FLM generates sequences by rotating vectors in mathbbSd-1 along a velocity field learned with cross-entropy, avoiding the overhead of materializing one-hot vectors. Previous FLMs match AR in Generative Perplexity (Gen. PPL), but samples with high likelihood are not necessarily correct in verifiable domains such as math and code. mathbbS-FLM substantially improves continuous flow language models on large-vocabulary reasoning and closes the gap to masked diffusion under standard-temperature sampling (T=1), while a gap remains under optimized low-temperature (T=0.1) decoding.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 32 of its 65 references that are in the map. First posted on arXiv 11 May 2026.

Builds on

32 of 65 references.

Closest ancestors · 8

Same direction, newest first

  1. 2026CoDAR: Continuous Diffusion Language Models are More Powerful Than You Think3 Mar
  2. 2025Scaling Behavior of Discrete Diffusion Language Models11 Dec
  3. 2025Loopholing Discrete Diffusion: Deterministic Bypass of the Sampling Wall22 Oct
  4. 2025Fine-Tuning Masked Diffusion for Provable Self-Correction1 Oct
  5. 2025Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmasking30 May
  6. 2025Theoretical Benefit and Limitation of Diffusion Language Model13 Feb
  7. 2024Scaling up Masked Diffusion Models on Text24 Oct
  8. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct

Other references · 24

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  6. 2020Scaling Laws for Neural Language Modelscited by 6,586
  7. 2022Scalable Diffusion Models with Transformerscited by 6,054
  8. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  9. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  10. 2018Neural Ordinary Differential Equationscited by 3,779
  11. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,405
  12. 2023Improving and generalizing flow-based generative models with minibatch optimal transportcited by 879
12 more
  1. 2018FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Modelscited by 594
  2. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 451
  3. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 420
  4. 2023Analyzing and Improving the Training Dynamics of Diffusion Modelscited by 402
  5. 2018Hyperspherical Variational Auto-Encoderscited by 189
  6. 2020Riemannian Continuous Normalizing Flowscited by 145
  7. 2024Unlocking Guidance for Discrete State-Space Diffusion and Flow Modelscited by 129
  8. 2020Neural Manifold Ordinary Differential Equationscited by 81
  9. 2018Spherical Latent Spaces for Stable Variational Autoencoderscited by 67
  10. 2025Roll the dice & look before you leap: Going beyond the creative limits of next-token predictioncited by 29
  11. 2024The Factorization Curse: Which Tokens You Predict Underlie the Reversal Curse and Morecited by 25
  12. 2024Arrows of Time for Large Language Modelscited by 20

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 22.

22 papers.

Sort:

By area:

  1. 2026Self-conditioned Flow Map Language Models via Fixed-point Flowscited by 14
  2. 2026Posterior Refinement: Fast Language Generation via Any-Order Flow Mapscited by 11
  3. 2026DiffusionGemma Technical Reportcited by 11
  4. 2026Thinking with Looped Flowscited by 9
  5. 2026Improving Neural Network Training by Decoupling the Magnitude and Direction of Weight Vectorscited by 3
  6. 2026Discrete Beckmann Transport Models for One-Step Language Modeling and Reasoningcited by 3
  7. 2026Simplex Relaxation for Discrete Diffusioncited by 2
  8. 2026Distilled Continuous Diffusion Language Models Can Write Code in Few Steps---or Onecited by 2
  9. 2026From Interface to Inference: Eliciting Any-Order Inference from Any-Order Modelscited by 1
  10. 2026ELF-REG: Scaling Continuous Diffusion Language Models to Reasoning Taskscited by 1
  11. 2026Simplex Diffusion Modelscited by 1
  12. 2026Reasoning with Continuous Latent Diffusioncited by 1
10 more
  1. 2026Alpha Diffusion Language Models: Factorization Alone Is Not the Problemcited by 0
  2. 2026Gumbel Straight Flow: Distilling Autoregressive Models into One-step Flow Mapscited by 0
  3. 2026Clock Diffusion: Efficient Semi-Autoregressive Continuous Diffusion Language Modelscited by 0
  4. 2026Jumping up and down: Denoiser diffusion models for discrete ordinal datacited by 0
  5. 2026Representation-Space MMD for Diffusion Language Modelscited by 0
  6. 2026Denoising Hierarchical Representations: Joint Continuous Diffusion for Language Modelingcited by 0
  7. 2026Noise Your Prompt: Noising Conditioning Tokens in Continuous Diffusion Language Modelscited by 0
  8. 2026Unrolled Flow Models for Reasoningcited by 0
  9. 2026On the Design Space of Continuous Diffusion Language Modelscited by 0
  10. 2026Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Flow Map Language Models: One-step Language Modeling via Continuous Denoising
  2. 2026Sampling from Flow Language Models via Marginal-Conditioned Bridges
  3. 2026Masked Language Flow Models
  4. 2026LangFlow: Continuous Diffusion Rivals Discrete in Language Modeling
  5. 2026Scaling Categorical Flow Maps
  6. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Models
  7. 2026Towards Latent Diffusion Suitable For Text
  8. 2023Likelihood-Based Diffusion Language Models