mlsanityData to 7 Oct 2026

Neuro-Inspired Inverse Learning for Planning and Control

arXiv 2605.24152 ↗First posted 22 May 2026cs.AIcited by 0

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

We present a neuro-inspired framework for embodied planning and control. Building on three principles that enable fast and highly effective goal-directed behavior in the mammalian brain - paired forward/inverse internal models, open-loop multi-step motor commands, and sequential, hierarchical organization of action - our Inverter framework uses learned components, trained end-to-end through Inverse Learning (IL) and supplemented where natural by analytic or algorithmic modules; we formalize IL and delineate it from supervised, reinforcement, and imitation learning. IL bridges Reinforcement Learning (RL)-style amortization, which runs in a single forward pass but emits only one action at a time, and Optimal Control (OC)-style sequence planning over whole trajectories, but with iterative test-time computation. Single Inverters or hierarchical n=2 Inverter stacks match or improve on offline-RL and diffusion-planner baselines on all 3 maze2d and 6 antmaze D4RL variants by an average of +24.2% (range -1.9% to +78.2%), at one-to-two orders of magnitude less inference compute time. Distinctively, optimizing through the Forward Model (FoM) over the entire T-step action sequence - rather than per step - lets Inverters produce smooth, goal-coherent, trajectory-wide structure and reach control policies closer to the analytic optimum than the policy underlying the training data itself. We also identify a failure mode of IL: FoM hacking under narrow training-data coverage, which we mitigate by using random training data with broader coverage. As an application example, a Pulse Inverter synthesizes arbitrary single-qubit quantum gates with fidelity matching the standard iterative numerical baseline (GRAPE), at more than 1000x lower per-gate compute time. In summary, we conclude that IL enables a versatile class of world-interfaces, especially for latency- and resource-critical embodied AI.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 44 of its 107 references that are in the map. First posted on arXiv 22 May 2026.

Builds on

44 of 107 references.

References · 44

  1. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  2. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  3. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  4. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 1,979
  5. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  6. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  7. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  8. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  9. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  10. 2022Diffusion Posterior Sampling for General Noisy Inverse Problemscited by 1,375
  11. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  12. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
32 more
  1. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  2. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
  3. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 882
  4. 2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 647
  5. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 636
  6. 2017OptNet: Differentiable Optimization as a Layer in Neural Networkscited by 588
  7. 2022Efficiently Scaling Transformer Inferencecited by 529
  8. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 482
  9. 2021Implicit Behavioral Cloningcited by 442
  10. 2023Directly Fine-Tuning Diffusion Models on Differentiable Rewardscited by 431
  11. 2021Uncertainty-Based Offline Reinforcement Learning with Diversified Q-Ensemblecited by 297
  12. 2020Deep Learning Techniques for Inverse Problems in Imagingcited by 290
  13. 2020Integrated Task and Motion Planningcited by 281
  14. 2022On Neural Differential Equationscited by 264
  15. 2024Adjoint Matching: Fine-tuning Flow and Diffusion Generative Models with Memoryless Stochastic Optimal Controlcited by 222
  16. 2018Differentiable MPC for End-to-end Planning and Controlcited by 178
  17. 2015Learning Continuous Control Policies by Stochastic Value Gradientscited by 156
  18. 2023Revisiting the Minimalist Approach to Offline Reinforcement Learningcited by 121
  19. 2022CORL: Research-oriented Deep Offline Reinforcement Learning Librarycited by 117
  20. 2020Hyperparameter Selection for Offline Reinforcement Learningcited by 113
  21. 2022When Should We Prefer Offline Reinforcement Learning Over Behavioral Cloning?cited by 86
  22. 2018Motion Planning Networkscited by 78
  23. 2025Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Modelscited by 77
  24. 2024Fine-Tuning Discrete Diffusion Models via Reward Optimization with Applications to DNA and Protein Designcited by 72
  25. 2022Efficient Planning in a Compact Latent Action Spacecited by 57
  26. 2023Adversarial Model for Offline Reinforcement Learningcited by 31
  27. 2019MPC-Net: A First Principles Guided Policy Searchcited by 28
  28. 2021Showing Your Offline Reinforcement Learning Work: Online Evaluation Budget Matterscited by 24
  29. 2024Hierarchical World Models as Visual Whole-Body Humanoid Controllerscited by 24
  30. 2019Warm Start of Mixed-Integer Programs for Model Predictive Control of Hybrid Systemscited by 17
  31. 2022Conservative Bayesian Model-Based Value Expansion for Offline Policy Optimizationcited by 16
  32. 2025A Clean Slate for Offline Reinforcement Learningcited by 10

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026ACID: Action Consistency via Inverse Dynamics for Planning with World Models
  2. 2020Neural Dynamic Policies for End-to-End Sensorimotor Learning
  3. 2025Few-Shot Neuro-Symbolic Imitation Learning for Long-Horizon Planning and Acting
  4. 2026Emergent Neural Automaton Policies: Learning Symbolic Structure from Visuomotor Trajectories ↗
  5. 2021Off-Dynamics Inverse Reinforcement Learning from Hetero-Domain
  6. 2026Breaking Task Impasses Quickly: Adaptive Neuro-Symbolic Learning for Open-World Robotics
  7. 2025Model Predictive Adversarial Imitation Learning for Planning from Observation
  8. 2021Inverse reinforcement learning for autonomous navigation via differentiable semantic mapping and planning