mlsanityData to 14 Oct 2026

DrDiff: Dynamic Routing Diffusion with Hierarchical Attention for Breaking the Efficiency-Quality Trade-off

arXiv 2509.02785 ↗First posted 2 Sep 2025cs.CL cs.AIcited by 34

Not on the map. Why?

Abstract (arXiv)

This paper introduces DrDiff, a novel framework for long-text generation that overcomes the efficiency-quality trade-off through three core technologies. First, we design a dynamic expert scheduling mechanism that intelligently allocates computational resources during the diffusion process based on text complexity, enabling more efficient handling of text generation tasks of varying difficulty. Second, we introduce a Hierarchical Sparse Attention (HSA) mechanism that adaptively adjusts attention patterns according to a variety of input lengths, reducing computational complexity from O(n2) to O(n) while maintaining model performance. Finally, we propose a soft absorption guidance optimization strategy that combines with DPM-solver++ to reduce diffusion steps, significantly improving generation speed. Comprehensive experiments on various long-text generation benchmarks demonstrate the superiority of our DrDiff over the existing SOTA methods.

Key facts. Cited by 34 papers in the map (data through 14 Oct 2026). Builds on 31 of its 46 references that are in the map. First posted on arXiv 2 Sep 2025.

Builds on

31 of 46 references.

References · 31

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  6. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  7. 2020Longformer: The Long-Document Transformercited by 3,413
  8. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  9. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  10. 2024Qwen2 Technical Reportcited by 2,120
  11. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,850
  12. 2020Reformer: The Efficient Transformercited by 1,668
19 more
  1. 2019Generating Long Sequences with Sparse Transformerscited by 1,546
  2. 2020Linformer: Self-Attention with Linear Complexitycited by 1,474
  3. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  4. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inferencecited by 501
  5. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Modelscited by 450
  6. 2017Constructing Datasets for Multi-hop Reading Comprehension Across Documentscited by 190
  7. 2023Walking Down the Memory Maze: Beyond Context Limit through Interactive Readingcited by 133
  8. 2023Unlimiformer: Long-Range Transformers with Unlimited Length Inputcited by 128
  9. 2024SeerAttention: Learning Intrinsic Sparse Attention in Your LLMscited by 124
  10. 2015Yara Parser: A Fast and Accurate Dependency Parsercited by 111
  11. 2024Energy-Based Diffusion Language Models for Text Generationcited by 89
  12. 2022Text Generation with Diffusion Language Models: A Pre-training Approach with Continuous Paragraph Denoisecited by 79
  13. 2022SeqDiffuSeq: Text Diffusion with Encoder-Decoder Transformerscited by 64
  14. 2025CF-VLM:CounterFactual Vision-Language Fine-tuningcited by 59
  15. 2025GAM-Agent: Game-Theoretic and Uncertainty-Aware Collaboration for Complex Visual Reasoningcited by 58
  16. 2021Long Text Generation by Modeling Sentence-Level and Discourse-Level Coherencecited by 45
  17. 2023Can Diffusion Model Achieve Better Performance in Text Generation? Bridging the Gap between Training and Inference!cited by 11
  18. 2025EDiT: Efficient Diffusion Transformers with Linear Compressed Attentioncited by 8
  19. 2025TokenSwift: Lossless Acceleration of Ultra Long Sequence Generationcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 25; 2026: 9.

34 papers.

Sort:

By area:

  1. 2025LongCodeZip: Compress Long Context for Code Language Modelscited by 39
  2. 2025MAT-Agent: Adaptive Multi-Agent Training Optimizationcited by 35
  3. 2025ContextFlow: Training-Free Video Object Editing via Adaptive Context Enrichmentcited by 27
  4. 2026FastVMT: Eliminating Redundancy in Video Motion Transfercited by 23
  5. 2025Learning Dynamics of VLM Finetuningcited by 17
  6. 2025Top-Down Semantic Refinement for Image Captioningcited by 16
  7. 2025LLM-CAS: Dynamic Neuron Perturbation for Real-Time Hallucination Correctioncited by 15
  8. 2025RaCoT: Plug-and-Play Contrastive Example Generation Mechanism for Enhanced LLM Reasoning Reliabilitycited by 14
  9. 2025Cost-Effective Communication: An Auction-based Method for Language Agent Interactioncited by 14
  10. 2025CTCC: A Robust and Stealthy Fingerprinting Framework for Large Language Models via Cross-Turn Contextual Correlation Backdoorcited by 12
  11. 20253DAlign-DAER: Dynamic Attention Policy and Efficient Retrieval Strategy for Fine-grained 3D-Text Alignment at Scalecited by 12
  12. 2025FlashVLM: Text-Guided Visual Token Selection for Large Multimodal Modelscited by 11
22 more
  1. 2025STORM: Search-Guided Generative World Models for Robotic Manipulationcited by 7
  2. 2025CoAgent: Collaborative Planning and Consistency Agent for Coherent Video Generationcited by 4
  3. 2025MultiMotion: Multi Subject Video Motion Transfer via Video Diffusion Transformercited by 3
  4. 2026AgriWorld:A World Tools Protocol Framework for Verifiable Agricultural Reasoning with Code-Executing LLM Agentscited by 2
  5. 2026ResWM: Residual-Action World Model for Visual RLcited by 2
  6. 2025MoE-DiffuSeq: Enhancing Long-Document Diffusion Models with Sparse Attention and Mixture of Expertscited by 1
  7. 2025Guardian: Decoupling Exploration from Safety in Reinforcement Learningcited by 0
  8. 2025PTTA: A Pure Text-to-Animation Framework for High-Quality Creationcited by 0
  9. 2025SirenPose: Dynamic Scene Reconstruction via Geometric Supervisioncited by 0
  10. 2025SA-DiffuSeq: Addressing Computational and Scalability Challenges in Long-Document Generation with Sparse Attentioncited by 0
  11. 2025RevFFN: Memory-Efficient Full-Parameter Fine-Tuning of Mixture-of-Experts LLMs with Reversible Blockscited by 0
  12. 2025A Three-Level Alignment Framework for Large-Scale 3D Retrieval and Controlled 4D Generationcited by 0
  13. 2025Learning Dynamic Scene Reconstruction with Sinusoidal Geometric Priorscited by 0
  14. 2025Towards Efficient Post-Training via Fourier-Driven Adapter Architecturescited by 0
  15. 2025Self-Rewarded Multimodal Coherent Reasoning Across Diverse Visual Domainscited by 0
  16. 20263D-Agent:Tri-Modal Multi-Agent Collaboration for Scalable 3D Object Annotationcited by 0
  17. 2025Forgetting as a Feature: Cognitive Alignment of Large Language Modelscited by 0
  18. 2026Multi-Agent Cooperative Learning for Robust Vision-Language Alignment under OOD Conceptscited by 0
  19. 2026Bridging the Semantic Chasm: Synergistic Conceptual Anchoring for Generalized Few-Shot and Zero-Shot OOD Perceptioncited by 0
  20. 2026A Scalable Curiosity-Driven Game-Theoretic Framework for Long-Tail Multi-Label Learning in Data Miningcited by 0
  21. 2026ProgAgent:A Continual RL Agent with Progress-Aware Rewardscited by 0
  22. 2026CacheScope: Exact-Compute Allocation for Training-Free Diffusion Accelerationcited by 0