# Memory, Benchmark & Robots: A Benchmark for Solving Complex Tasks with Reinforcement Learning

arXiv: https://arxiv.org/abs/2502.10550 · HTML: https://mlsanity.com/paper/2502.10550/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 71 of its 96 references that are in the map.
- First posted on arXiv 14 Feb 2025.

## Placed in

- Deep reinforcement learning methods / [Deep RL algorithms and representations](https://mlsanity.com/deeprl/deep-rl-algorithms-and-representations/)

## Builds on (most cited first, 30 of 71)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Conservative Q-Learning for Offline Reinforcement Learning](https://mlsanity.com/paper/2006.04779/) (2020)
- [Graph Neural Networks: A Review of Methods and Applications](https://mlsanity.com/paper/1812.08434/) (2018)
- [Decision Transformer: Reinforcement Learning via Sequence Modeling](https://mlsanity.com/paper/2106.01345/) (2021)
- [LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding](https://mlsanity.com/paper/2308.14508/) (2023)
- [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018)
- [Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning](https://mlsanity.com/paper/1910.10897/) (2019)
- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [AI2-THOR: An Interactive 3D Environment for Visual AI](https://mlsanity.com/paper/1712.05474/) (2017)
- [Simplified State Space Layers for Sequence Modeling](https://mlsanity.com/paper/2208.04933/) (2022)
- [RLBench: The Robot Learning Benchmark & Learning Environment](https://mlsanity.com/paper/1909.12271/) (2019)
- [CLIPort: What and Where Pathways for Robotic Manipulation](https://mlsanity.com/paper/2109.12098/) (2021)
- [TD-MPC2: Scalable, Robust World Models for Continuous Control](https://mlsanity.com/paper/2310.16828/) (2023)
- [robosuite: A Modular Simulation Framework and Benchmark for Robot Learning](https://mlsanity.com/paper/2009.12293/) (2020)
- [Habitat 2.0: Training Home Assistants to Rearrange their Habitat](https://mlsanity.com/paper/2106.14405/) (2021)
- [Introduction to Multi-Armed Bandits](https://mlsanity.com/paper/1904.07272/) (2019)
- [Deep Recurrent Q-Learning for Partially Observable MDPs](https://mlsanity.com/paper/1507.06527/) (2015)
- [VIMA: General Robot Manipulation with Multimodal Prompts](https://mlsanity.com/paper/2210.03094/) (2022)
- [RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robots](https://mlsanity.com/paper/2406.02523/) (2024)
- [Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learning](https://mlsanity.com/paper/1910.11956/) (2019)
- [dm_control: Software and Tasks for Continuous Control](https://mlsanity.com/paper/2006.12983/) (2020)
- [Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Models](https://mlsanity.com/paper/2402.07865/) (2024)
- [Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection](https://mlsanity.com/paper/1603.02199/) (2016)
- [Stabilizing Transformers for Reinforcement Learning](https://mlsanity.com/paper/1910.06764/) (2019)

## Built on by (most cited first, 30 of 36)

- [MemoryVLA: Perceptual-Cognitive Memory in Vision-Language-Action Models for Robotic Manipulation](https://mlsanity.com/paper/2508.19236/) (2025)
- [A Survey of Context Engineering for Large Language Models](https://mlsanity.com/paper/2507.13334/) (2025)
- [Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Survey](https://mlsanity.com/paper/2508.13073/) (2025)
- [RMBench: Memory-Dependent Robotic Manipulation Benchmark with Insights into Policy Design](https://mlsanity.com/paper/2603.01229/) (2026)
- [Don't Blind Your VLA: Aligning Visual Representations for OOD Generalization](https://mlsanity.com/paper/2510.25616/) (2025)
- [EventVLA: Event-Driven Visual Evidence Memory for Long-Horizon Vision-Language-Action Policies](https://mlsanity.com/paper/2606.20092/) (2026)
- [Rethinking Progression of Memory State in Robotic Manipulation: An Object-Centric Perspective](https://mlsanity.com/paper/2511.11478/) (2025)
- [Gated Memory Policy: In-Context Memorization and Adaptation](https://mlsanity.com/paper/2604.18933/) (2026)
- [MemoryVLA++: Temporal Modeling via Memory and Imagination in Vision-Language-Action Models](https://mlsanity.com/paper/2606.09827/) (2026)
- [μVLA: On Recurrent Memory for Partially Observable Manipulation in VLA Models](https://mlsanity.com/paper/2606.12497/) (2026)
- [vla-eval: A Unified Evaluation Harness for Vision-Language-Action Models](https://mlsanity.com/paper/2603.13966/) (2026)
- [ELMUR: External Layer Memory with Update/Rewrite for Long-Horizon RL Problems](https://mlsanity.com/paper/2510.07151/) (2025)
- [NEBULA: Do We Evaluate Vision-Language-Action Agents Correctly?](https://mlsanity.com/paper/2510.16263/) (2025)
- [VPWEM: Non-Markovian Visuomotor Policy with Working and Episodic Memory](https://mlsanity.com/paper/2603.04910/) (2026)
- [Data Pyramid for Embodied Manipulation: A Survey](https://mlsanity.com/paper/2607.24744/) (2026)
- [Chameleon: Control-Indexed Prospective Memory for Visuomotor Manipulation](https://mlsanity.com/paper/2603.24576/) (2026)
- [SimpleMemVLA: A Simple but Effective Native-Video Memory for Vision-Language-Action Models](https://mlsanity.com/paper/2609.05533/) (2026)
- [WatchAct: A Benchmark for Behavior-Grounded Robot Manipulation](https://mlsanity.com/paper/2606.26443/) (2026)
- [Memory Retention Is Not Enough to Master Memory Tasks in Reinforcement Learning](https://mlsanity.com/paper/2601.15086/) (2026)
- [FibVLA: An Efficient Temporal Vision-Language-Action Model with Fibonacci Sampling](https://mlsanity.com/paper/2607.29596/) (2026)
- [RoboSPA: Can VLA Models Go Beyond Simple Scenes and Short-Horizon Tasks?](https://mlsanity.com/paper/2609.05324/) (2026)
- [MEMOBench: A Process Level Memory Benchmark for Robotic Manipulation](https://mlsanity.com/paper/2609.07047/) (2026)
- [MemBodied: Recurrent Associative Memory for Vision-Language-Action Models](https://mlsanity.com/paper/2609.28256/) (2026)
- [In-Context Learning for Robots: Methods and Applications](https://mlsanity.com/paper/2609.36012/) (2026)
- [KAGE-Bench: Fast Known-Axis Visual Generalization Evaluation for Reinforcement Learning](https://mlsanity.com/paper/2601.14232/) (2026)
- [Does VLA Even Know the Basics? Measuring Commonsense and World Knowledge Retention in Vision-Language-Action Models](https://mlsanity.com/paper/2606.19297/) (2026)
- [Learning to Use Imagination: Progress-Conditioned Future Utilization for World Action Models](https://mlsanity.com/paper/2609.06578/) (2026)
- [SmoLSTM: A Compact Vision-Language-Action Model with Recurrent Memory that Persists](https://mlsanity.com/paper/2609.22854/) (2026)
- [ActiveArena: Benchmarking and Understanding Active Perception in Robotic Manipulation](https://mlsanity.com/paper/2609.24124/) (2026)
- [Minimal Recurrent Behavioral Memory for Imitation under Partial Observability](https://mlsanity.com/paper/2609.25757/) (2026)

## Abstract (arXiv)

Memory is crucial for enabling agents to tackle complex tasks with temporal and spatial dependencies. While many reinforcement learning (RL) algorithms incorporate memory, the field lacks a universal benchmark to assess an agent's memory capabilities across diverse scenarios. This gap is particularly evident in tabletop robotic manipulation, where memory is essential for solving tasks with partial observability and ensuring robust performance, yet no standardized benchmarks exist. To address this, we introduce MIKASA (Memory-Intensive Skills Assessment Suite for Agents), a comprehensive benchmark for memory RL, with three key contributions: (1) we propose a comprehensive classification framework for memory-intensive RL tasks, (2) we collect MIKASA-Base -- a unified benchmark that enables systematic evaluation of memory-enhanced agents across diverse scenarios, and (3) we develop MIKASA-Robo (pip install mikasa-robo-suite) -- a novel benchmark of 32 carefully designed memory-intensive tasks that assess memory capabilities in tabletop robotic manipulation. Our work introduces a unified framework to advance memory RL research, enabling more robust systems for real-world use. MIKASA is available at https://tinyurl.com/membenchrobots.