mlsanityData to 7 Oct 2026

HOFAR: High-Order Augmentation of Flow Autoregressive Transformers

arXiv 2503.08032 ↗First posted 11 Mar 2025cs.CV cs.AI cs.LGcited by 4

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Flow Matching and Transformer architectures have demonstrated remarkable performance in image generation tasks, with recent work FlowAR [Ren et al., 2024] synergistically integrating both paradigms to advance synthesis fidelity. However, current FlowAR implementations remain constrained by first-order trajectory modeling during the generation process. This paper introduces a novel framework that systematically enhances flow autoregressive transformers through high-order supervision. We provide theoretical analysis and empirical evaluation showing that our High-Order FlowAR (HOFAR) demonstrates measurable improvements in generation quality compared to baseline models. The proposed approach advances the understanding of flow-based autoregressive modeling by introducing a systematic framework for analyzing trajectory dynamics through high-order expansion.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 51 of its 55 references that are in the map. First posted on arXiv 11 Mar 2025.

Builds on

51 of 55 references.

Closest ancestors · 10

Same direction, newest first

  1. 2024LazyDiT: Lazy Learning for the Acceleration of Diffusion Transformers17 Dec
  2. 2024On Statistical Rates of Conditional Diffusion Transformers: Approximation, Estimation and Minimax Optimality26 Nov
  3. 2024Unraveling the Smoothness Properties of Diffusion Models: A Gaussian Mixture Perspective26 May
  4. 2024Exploring the Frontiers of Softmax: Provable Optimization, Applications in Diffusion Model, and Beyond6 May
  5. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesis5 Mar
  6. 2023NoiseCLR: A Contrastive Learning Approach for Unsupervised Discovery of Interpretable Directions in Diffusion Models8 Dec
  7. 2023On Exact Inversion of DPM-Solvers30 Nov
  8. 2023Simple diffusion: End-to-end diffusion for high resolution images26 Jan
  9. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  10. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 41

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  3. 2022Flow Matching for Generative Modelingcited by 5,426
  4. 2018Neural Ordinary Differential Equationscited by 3,733
  5. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  6. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  7. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,005
  8. 2024Autoregressive Image Generation without Vector Quantizationcited by 644
  9. 2018FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Modelscited by 584
  10. 2024Pyramidal Flow Matching for Efficient Video Generative Modelingcited by 354
  11. 2023MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Modelcited by 330
  12. 2020How to train your neural ODE: the world of Jacobian and kinetic regularizationcited by 228
29 more
  1. 20244Diffusion: Multi-view Video Diffusion Model for 4D Generationcited by 104
  2. 2024Diffusion4D: Fast Spatial-temporal Consistent 4D Generation via Video Diffusion Modelscited by 99
  3. 2023HumanNorm: Learning Normal Diffusion Model for High-quality and Realistic 3D Human Generationcited by 88
  4. 2020Learning Differential Equations that are Easy to Solvecited by 81
  5. 2023Beyond First-Order Tweedie: Solving Inverse Problems using Latent Diffusioncited by 78
  6. 2024FlowAR: Scale-wise Autoregressive Image Generation Meets Flow Matchingcited by 77
  7. 2023VideoGen: A Reference-Guided Latent Diffusion Approach for High Definition Text-to-Video Generationcited by 74
  8. 2024Video Interpolation with Diffusion Modelscited by 67
  9. 2023TokenCompose: Text-to-Image Diffusion with Token-level Supervisioncited by 40
  10. 2023DiffPortrait3D: Controllable Diffusion for Zero-Shot Portrait View Synthesiscited by 26
  11. 2024OmniControlNet: Dual-stage Integration for Conditional Image Generationcited by 24
  12. 2023Dolfin: Diffusion Layout Transformers without Autoencodercited by 23
  13. 2024Score-based Generative Diffusion Models for Social Recommendationscited by 23
  14. 2024Multi-Layer Transformers Gradient Can be Approximated in Almost Linear Timecited by 22
  15. 2024Gen-3Diffusion: Realistic Image-to-3D Generation via 2D & 3D Diffusion Synergycited by 16
  16. 2025High-Order Matching for One-Step Shortcut Diffusion Modelscited by 15
  17. 2024Differential Privacy Mechanisms in Neural Tangent Kernel Regressioncited by 14
  18. 2025On Computational Limits and Provably Efficient Criteria of Visual Autoregressive Models: A Fine-Grained Complexity Analysiscited by 14
  19. 2022Taylor-Lagrange Neural Ordinary Differential Equations: Toward Fast Training and Evaluation of Neural ODEscited by 13
  20. 2024FreeDiff: Progressive Frequency Truncation for Image Editing with Diffusion Modelscited by 13
  21. 2025Force Matching with Relativistic Constraints: A Physics-Inspired Approach to Stable and Efficient Generative Modelingcited by 12
  22. 2025On Computational Limits of FlowAR Models: Expressivity and Efficiencycited by 10
  23. 2025Dissecting Submission Limit in Desk-Rejections: A Mathematical Analysis of Fairness in AI Conference Policiescited by 9
  24. 2022Sublinear Time Algorithm for Online Weighted Bipartite Matchingcited by 7
  25. 2023Fast and Efficient Matching Algorithm with Deadline Instancescited by 7
  26. 2024Efficient 3D Shape Generation via Diffusion Mamba with Bidirectional SSMscited by 7
  27. 2025ARFlow: Autoregressive Flow with Hybrid Linear Attentioncited by 7
  28. 2025RichSpace: Enriching Text-to-Video Prompt Space via Text Embedding Interpolationcited by 6
  29. 2025DPBloomfilter: Securing Bloom Filters with Differential Privacycited by 1

Built on by

Cited by year

Citing papers per year: 2025: 4.

4 papers.

  1. 2025T2VWorldBench: A Benchmark for Evaluating World Knowledge in Text-to-Video Generationcited by 18
  2. 2025T2VTextBench: A Human Evaluation Benchmark for Textual Control in Video Generation Modelscited by 9
  3. 2025Towards High-Order Mean Flow Generative Models: Feasibility, Expressivity, and Provably Efficient Criteriacited by 5
  4. 2025Fundamental Limits of Crystalline Equivariant Graph Neural Networks: A Circuit Complexity Perspectivecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Universal Approximation of Visual Autoregressive Transformers
  2. 2025NAMI: Efficient Image Generation via Bridged Progressive Rectified Flow Transformers
  3. 2024Revisiting Non-Autoregressive Transformers for Efficient Image Synthesis
  4. 2025StreamFlow: Theory, Algorithm, and Implementation for High-Efficiency Rectified Flow Generation
  5. 2025Nested AutoRegressive Models
  6. 2025Autoregressive Distillation of Diffusion Transformers
  7. 2026Training-Free Hidden-State Refinement for Flow-Matching Image Generators
  8. 2026LayerCache: Exploiting Layer-wise Velocity Heterogeneity for Efficient Flow Matching Inference