MIRTH: Mutual-Information Reasoning with Temporal Hubs for Vision-Language-Action Agents
On the map: Vision-language-action models → Efficient VLA inference and adaptation
Abstract (arXiv)
VLA models have emerged as a powerful paradigm for transferring semantic knowledge from web-scale data to physical robotic control. However, current single-frame architectures suffer from intrinsic limitations: temporal myopia that discards historical dynamics, reasoning gaps between high-level instructions and low-level motor commands, and inference inefficiency due to autoregressive scalar decoding. In this work, we propose MIRTH, a unified framework designed to address these challenges. MIRTH augments a pretrained VLA backbone with three key innovations: (1) dual-scale temporal memory hubs that compress long-term scene evolution and short-term motion trends into compact embeddings; (2) latent reasoning tokens optimized via a mutual-information objective carving out a semantic plan space to align multimodal context with action trajectories; and (3) a parallel action decoding scheme that replaces autoregressive generation with vector-wise prediction to maximize control throughput. Extensive evaluations on the LIBERO simulation benchmark and a real-world LeRobot platform demonstrate that MIRTH achieves state-of-the-art performance and exhibiting emergent error recovery capabilities. The codes and collected datasets are released at http://github.com/kiva12138/mirth.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Efficient VLA inference and adaptation, one of 13 directions in Vision-language-action models. Builds on 17 of its 23 references that are in the map. First posted on arXiv 30 Jun 2026.
Builds on
17 of 23 references.
References · 17
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
- 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
- 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
- 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,210
- 2021Multiscale Vision Transformerscited by 897
- 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Successcited by 883
- 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 750
- 2014Memory Networkscited by 363
- 2023Vision-Language Foundation Models as Effective Robot Imitatorscited by 349
5 more
- 2018Long-Term Feature Banks for Detailed Video Understandingcited by 239
- 2023LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agentscited by 193
- 2023Q-Transformer: Scalable Offline Reinforcement Learning via Autoregressive Q-Functionscited by 131
- 2024PO-GUISE+: Pose and object guided transformer token selection for efficient driver action recognitioncited by 2
- 2025HELIOS: Hierarchical Exploration for Language-Grounded Interaction in Open Scenescited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026ElegantVLA: Learning When to Think for Efficient Vision-Language-Action Models
- 2026G0.5: One Autoregressive Stream for Robot Reasoning and Action
- 2025Think Twice, Act Once: Token-Aware Compression and Action Reuse for Efficient Inference in Vision-Language-Action Models
- 2026Explicit Language Memory for Long-Horizon Planning in Vision-Language-Action Models
- 2026Towards Long-horizon Embodied Agents with Tool-Aligned Vision-Language-Action Models
- 2025VLA-Cache: Efficient Vision-Language-Action Manipulation via Adaptive Token Caching
- 2026PHR-VLA: Planning Horizon Reasoning for Vision-Language-Action Models
- 2026AR-VLA: True Autoregressive Action Expert for Vision-Language-Action Models