mlsanityData to 14 Oct 2026

Memory-Consistent Neural Networks for Imitation Learning

arXiv 2310.06171 ↗First posted 9 Oct 2023cs.LG cs.AI cs.ROcited by 14

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

Imitation learning considerably simplifies policy synthesis compared to alternative approaches by exploiting access to expert demonstrations. For such imitation policies, errors away from the training samples are particularly critical. Even rare slip-ups in the policy action outputs can compound quickly over time, since they lead to unfamiliar future states where the policy is still more likely to err, eventually causing task failures. We revisit simple supervised ``behavior cloning'' for conveniently training the policy from nothing more than pre-recorded demonstrations, but carefully design the model class to counter the compounding error phenomenon. Our ``memory-consistent neural network'' (MCNN) outputs are hard-constrained to stay within clearly specified permissible regions anchored to prototypical ``memory'' training samples. We provide a guaranteed upper bound for the sub-optimality gap induced by MCNN policies. Using MCNNs on 10 imitation learning tasks, with MLP, Transformer, and Diffusion backbones, spanning dexterous robotic manipulation and driving, proprioceptive inputs and visual inputs, and varying sizes and types of demonstration data, we find large and consistent gains in performance, validating that MCNNs are better-suited than vanilla deep neural networks for imitation learning applications. Website: https://sites.google.com/view/mcnn-imitation

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 33 of its 50 references that are in the map. First posted on arXiv 9 Oct 2023.

Builds on

33 of 50 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware23 Apr
  2. 2022Behavior Transformers: Cloning k modes with one stone22 Jun
  3. 2021The Surprising Effectiveness of Representation Learning for Visual Imitation2 Dec
  4. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learning25 Oct

Other references · 29

  1. 2017Neural Discrete Representation Learningcited by 4,712
  2. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  3. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  4. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  5. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  6. 2016Generative Adversarial Imitation Learningcited by 1,597
  7. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,427
  8. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 899
  9. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 782
  10. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 725
  11. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 715
  12. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learningcited by 562
17 more
  1. 2021Implicit Behavioral Cloningcited by 452
  2. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 436
  3. 2019Exploring the Limitations of Behavior Cloning for Autonomous Drivingcited by 346
  4. 2022Transformers are Sample-Efficient World Modelscited by 328
  5. 2023Imitating Human Behaviour with Diffusion Modelscited by 263
  6. 2017DART: Noise Injection for Robust Imitation Learningcited by 191
  7. 2014Model-based Reinforcement Learning and the Eluder Dimensioncited by 116
  8. 2020Toward the Fundamental Limits of Imitation Learningcited by 96
  9. 2017Towards Generalization and Simplicity in Continuous Controlcited by 68
  10. 2023When Do Transformers Shine in RL? Decoupling Memory from Credit Assignmentcited by 63
  11. 2022Bisimulation Makes Analogies in Goal-Conditioned Reinforcement Learningcited by 38
  12. 2018Q-learning with Nearest Neighborscited by 37
  13. 2022iDECODe: In-distribution Equivariance for Conformal Out-of-distribution Detectioncited by 27
  14. 2021Improving Neural Network Robustness via Persistency of Excitationcited by 14
  15. 2022Guaranteed Conformance of Neurosymbolic Models to Natural Constraintscited by 6
  16. 2022Exploring with Sticky Mittens: Reinforcement Learning with Expert Interventions via Option Templatescited by 2
  17. 2022Memory Classifiers: Two-stage Classification for Robustness in Machine Learningcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 7; 2025: 5; 2026: 2.

14 papers.

Sort:

By area:

  1. 20243D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representationscited by 116
  2. 2024Policy Decorator: Model-Agnostic Online Refinement for Large Policy Modelcited by 70
  3. 2024Streaming Diffusion Policy: Fast Policy Synthesis with Variable Noise Diffusion Modelscited by 56
  4. 2024Affordance-based Robot Manipulation with Flow Matchingcited by 54
  5. 2025RICL: Adding In-Context Adaptability to Pre-Trained Vision-Language-Action Modelscited by 22
  6. 2024Deep Generative Models for Offline Policy Learning: Tutorial, Survey, and Perspectives on Future Directionscited by 20
  7. 2024REGENT: A Retrieval-Augmented Generalist Agent That Can Act In-Context in New Environmentscited by 16
  8. 2025CDP: Towards Robust Autoregressive Visuomotor Policy Learning via Causal Diffusioncited by 8
  9. 20254D Visual Pre-training for Robot Learningcited by 8
  10. 2025Sample from What You See: Visuomotor Policy Learning via Diffusion Bridge with Observation-Embedded Stochastic Differential Equationcited by 3
  11. 2026X-Distill: Cross-Architecture Vision Distillation for Visuomotor Learningcited by 2
  12. 2024Automating Weak Label Generation for Data Programming with Clinicians in the Loopcited by 1
2 more
  1. 2025SeFA-Policy: Fast and Accurate Visuomotor Policy Learning with Selective Flow Alignmentcited by 0
  2. 2026RE4: Transformation-aware Imitation of Object Interactions Using Manipulation Modescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Difference-Aware Retrieval Policies for Imitation Learning
  2. 2025Imitation Learning in the Deep Learning Era: A Novel Taxonomy and Recent Advances
  3. 2022Diverse Imitation Learning via Self-Organizing Generative Models
  4. 2026Where Small Mistakes Matter: Quantifying State Criticality for Imitation Learning
  5. 2024Efficient Imitation Learning with Conservative World Models
  6. 2026ReGIL: Retrieval-Guided Imitation Learning from a Single Demonstration
  7. 2022Learning and Retrieval from Prior Data for Skill-based Imitation Learning
  8. 2025Generalization Capability for Imitation Learning