# Ada-Diffuser: Latent-Aware Adaptive Diffusion for Decision-Making

arXiv: https://arxiv.org/abs/2605.16054 · HTML: https://mlsanity.com/paper/2605.16054/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models.
- Builds on 30 of its 98 references that are in the map.
- First posted on arXiv 15 May 2026.

## Placed in

- Diffusion models / [Diffusion and flow policies for robotics](https://mlsanity.com/diffusion/diffusion-and-flow-policies-for-robotics/)

## Builds on (most cited first, 30 of 30)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [D4RL: Datasets for Deep Data-Driven Reinforcement Learning](https://mlsanity.com/paper/2004.07219/) (2020)
- [Planning with Diffusion for Flexible Behavior Synthesis](https://mlsanity.com/paper/2205.09991/) (2022)
- [Open-Sora: Democratizing Efficient Video Production for All](https://mlsanity.com/paper/2412.20404/) (2024)
- [Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learning](https://mlsanity.com/paper/2208.06193/) (2022)
- [Is Conditional Generative Modeling all you need for Decision-Making?](https://mlsanity.com/paper/2211.15657/) (2022)
- [Temporal Difference Learning for Model Predictive Control](https://mlsanity.com/paper/2203.04955/) (2022)
- [Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasets](https://mlsanity.com/paper/2504.02792/) (2025)
- [Contextual Markov Decision Processes](https://mlsanity.com/paper/1502.02259/) (2015)
- [Unified Video Action Model](https://mlsanity.com/paper/2503.00200/) (2025)
- [AdaptDiffuser: Diffusion Models as Adaptive Self-evolving Planners](https://mlsanity.com/paper/2302.01877/) (2023)
- [Compositional Foundation Models for Hierarchical Planning](https://mlsanity.com/paper/2309.08587/) (2023)
- [Diffusion Models for Reinforcement Learning: A Survey](https://mlsanity.com/paper/2311.01223/) (2023)
- [Progressive Autoregressive Video Diffusion Models](https://mlsanity.com/paper/2410.08151/) (2024)
- [Temporal Difference Variational Auto-Encoder](https://mlsanity.com/paper/1806.03107/) (2018)
- [AdaRL: What, Where, and How to Adapt in Transfer Reinforcement Learning](https://mlsanity.com/paper/2107.02729/) (2021)
- [Re-Mix: Optimizing Data Mixtures for Large Scale Imitation Learning](https://mlsanity.com/paper/2408.14037/) (2024)
- [Zigzag Diffusion Sampling: Diffusion Models Can Self-Improve via Self-Reflection](https://mlsanity.com/paper/2412.10891/) (2024)
- [Neural Predictive Belief Representations](https://mlsanity.com/paper/1811.06407/) (2018)
- [Boosting Continuous Control with Consistency Policy](https://mlsanity.com/paper/2310.06343/) (2023)
- [Efficient Data Collection for Robotic Manipulation via Compositional Generalization](https://mlsanity.com/paper/2403.05110/) (2024)
- [ViD-GPT: Introducing GPT-style Autoregressive Generation in Video Diffusion Models](https://mlsanity.com/paper/2406.10981/) (2024)
- [Learning General Causal Structures with Hidden Dynamic Process for Climate Analysis](https://mlsanity.com/paper/2501.12500/) (2025)
- [Towards Empowerment Gain through Causal Structure Learning in Model-Based RL](https://mlsanity.com/paper/2502.10077/) (2025)
- [DynaMITE-RL: A Dynamic Model for Improved Temporal Meta-Reinforcement Learning](https://mlsanity.com/paper/2402.15957/) (2024)
- [Modeling Unseen Environments with Language-guided Composable Causal Components in Reinforcement Learning](https://mlsanity.com/paper/2505.08361/) (2025)
- [Stackelberg POMDP: A Reinforcement Learning Approach for Economic Design](https://mlsanity.com/paper/2210.03852/) (2022)

## Built on by (most cited first, 1 of 1)

- [Back to Parsimonious Latents: Learning Task-Centric World Models from Visual Foundations](https://mlsanity.com/paper/2605.25620/) (2026)

## Abstract (arXiv)

Recent work has framed decision-making as a sequence modeling problem using generative models such as diffusion models. Although promising, these approaches often overlook latent factors that exhibit evolving dynamics, elements that are fundamental to environment transitions, reward structures, and high-level agent behavior. Explicitly modeling these hidden processes is essential for both precise dynamics modeling and effective decision-making. In this paper, we propose a unified framework that explicitly incorporates latent dynamic inference into generative decision-making from minimal yet sufficient observations. We theoretically show that under mild conditions, the latent process can be identified from small temporal blocks of observations. Building on this insight, we introduce Ada-Diffuser, a causal diffusion model that learns the temporal structure of observed interactions and the underlying latent dynamics simultaneously, and furthermore, leverages them for planning and control. With a modular design, Ada-Diffuser supports both planning and policy learning tasks, enabling adaptation to latent variations in dynamics, rewards, and latent actions. Experiments on simulated control and robotic benchmarks demonstrate its effectiveness in accurate latent inference and adaptive policy learning.