mlsanityData to 7 Oct 2026

Adaptive Action Chunking via Multi-Chunk Q Value Estimation

arXiv 2605.10044 ↗First posted 11 May 2026cs.LG cs.AIcited by 4

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

Action chunking emerged as a pivotal technique in imitation learning, enabling policies to predict cohesive action sequences rather than single actions. Recently, this approach has expanded to reinforcement learning (RL), enhancing behavioral consistency and reducing bootstrapping errors in value function estimation. However, existing methods rely on a fixed chunk length, creating a performance bottleneck as the optimal length varies across states and tasks. In this paper, we propose Adaptive Action CHunking (ACH), a novel offline-to-online RL algorithm that dynamically modulates chunk length during both training and inference. To find the optimal chunk length for a dynamically varying current state, we simultaneously estimate action-values for all candidate chunk lengths in a single forward pass, using a Transformer-based architecture. Our mechanism allows the agent to select the most effective chunk length adaptively based on the current state. Evaluated on 34 challenging tasks, ACH consistently outperforms fixed-length baselines, demonstrating superior generalization and learning efficiency in complex environments.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 33 of its 38 references that are in the map. First posted on arXiv 11 May 2026.

Builds on

33 of 38 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware23 Apr

Other references · 32

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  4. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  5. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  6. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  7. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  8. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  9. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  10. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
  11. 2022Building Normalizing Flows with Stochastic Interpolantscited by 964
  12. 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Successcited by 844
20 more
  1. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  2. 2023Efficient Online Reinforcement Learning with Offline Datacited by 332
  3. 2023IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policiescited by 307
  4. 2023Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuningcited by 230
  5. 2024OGBench: Benchmarking Offline Goal-Conditioned RLcited by 216
  6. 2021Offline-to-Online Reinforcement Learning via Balanced Replay and Pessimistic Q-Ensemblecited by 211
  7. 2025Real-Time Execution of Action Chunking Flow Policiescited by 199
  8. 2024Efficient Online Reinforcement Learning Fine-Tuning Need Not Retain Offline Datacited by 58
  9. 2025Leave No Observation Behind: Real-time Correction for VLA Action Chunkscited by 45
  10. 2025EXPO: Stable Reinforcement Learning with Expressive Policiescited by 30
  11. 2024TOP-ERL: Transformer-based Off-Policy Episodic Reinforcement Learningcited by 18
  12. 2025Chunking the Critic: A Transformer-based Soft Actor-Critic with N-Step Returnscited by 16
  13. 2023A Perspective of Q-value Estimation on Offline-to-Online Reinforcement Learningcited by 15
  14. 2025Actor-Critic for Continuous Action Chunks: A Reinforcement Learning Framework for Long-Horizon Robotic Manipulation with Sparse Rewardcited by 13
  15. 2025Scalable Offline Model-Based RL with Action Chunkscited by 12
  16. 2024Coarse-to-fine Q-Network with Action Sequence for Data-Efficient Reinforcement Learningcited by 10
  17. 2024Optimistic Critic Reconstruction and Constrained Fine-Tuning for General Offline-to-Online RLcited by 9
  18. 2025DEAS: DEtached value learning with Action Sequence for Scalable Offline RLcited by 9
  19. 2026Flow Actor-Critic for Offline Reinforcement Learningcited by 6
  20. 2026SEAR: Sample Efficient Action Chunking Reinforcement Learningcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026SparkVLA: Stop-Aware Hierarchical VLA with Adaptive Action Chunking for Long-Horizon Manipulationcited by 1
  2. 2026Three Steps at a Time: Learning Representations from Action Sequences in Contrastive RLcited by 1
  3. 2026Dynamic Manipulation with World-Action Models via Counterfactual Planningcited by 0
  4. 2026FlexiWorld: Learning and Planning via Flexible Action Chunks Across Multiple Time Scalescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Reinforcement Learning with Action Chunking
  2. 2025Temporal Action Selection for Action Chunking
  3. 2026Adaptive Q-Chunking for Offline-to-Online Reinforcement Learning
  4. 2026ACSAC: Adaptive Chunk Size Actor-Critic with Causal Transformer Q-Network
  5. 2026Adaptive Action Chunking at Inference-time for Vision-Language-Action Models
  6. 2026Learning from Mixed-Quality Deployment Experience for Robot Manipulation
  7. 2026Act More, Decide Less: Skill-Guided Adaptive Action Chunking for Long-Horizon LLM Agents
  8. 2025CO-RFT: Efficient Fine-Tuning of Vision-Language-Action Models through Chunked Offline Reinforcement Learning