mlsanityData to 7 Oct 2026

DiSA-IQL: Offline Reinforcement Learning for Robust Soft Robot Control under Distribution Shifts

arXiv 2510.00358 ↗First posted 30 Sep 2025cs.RO cs.AIcited by 0

On the map: Offline RL and imitation learning from data → Offline RL algorithms

Abstract (arXiv)

Soft snake robots offer remarkable flexibility and adaptability in complex environments, yet their control remains challenging due to highly nonlinear dynamics. Existing model-based and bio-inspired controllers rely on simplified assumptions that limit their performance. Deep reinforcement learning (DRL) has recently emerged as a promising alternative, but online training is often impractical because of costly and potentially damaging real-world interactions. Offline RL provides a safer option by leveraging pre-collected datasets, but it suffers from distribution shift, which degrades generalization to unseen scenarios. To overcome this challenge, we propose DiSA-IQL (Distribution-Shift-Aware Implicit Q-Learning), an extension of IQL that incorporates robustness modulation by penalizing unreliable state-action pairs to mitigate distribution shift. We evaluate DiSA-IQL on goal-reaching tasks across two settings: in-distribution and out-of-distribution evaluation. Simulation results show that DiSA-IQL consistently outperforms baseline models, including Behavior Cloning (BC), Conservative Q-Learning (CQL), and vanilla IQL, achieving higher success rates, smoother trajectories, and greater robustness.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 14 of its 29 references that are in the map. First posted on arXiv 30 Sep 2025.

Builds on

14 of 29 references.

Closest ancestors · 7

Same direction, newest first

  1. 2023Bridging Distributionally Robust Learning and Offline RL: An Approach to Mitigate Distribution Shift and Partial Data Coverage27 Oct
  2. 2022Distributionally Robust Offline Reinforcement Learning with Linear Function Approximation14 Sep
  3. 2021Offline Reinforcement Learning with Implicit Q-Learning12 Oct
  4. 2021S4RL: Surprisingly Simple Self-Supervision for Offline Reinforcement Learning10 Mar
  5. 2020Conservative Q-Learning for Offline Reinforcement Learning8 Jun
  6. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems4 May
  7. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learning15 Apr

Other references · 7

  1. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  2. 2022DROPO: Sim-to-Real Transfer with Offline Domain Randomizationcited by 16
  3. 2023Data-driven Methods Applied to Soft Robot Modeling and Control: A Reviewcited by 11
  4. 2024Reinforcement Learning Controllers for Soft Robots using Learned Environmentscited by 3
  5. 2024Back-stepping Experience Replay with Application to Model-free Reinforcement Learning for a Soft Snake Robotcited by 2
  6. 2020Lateral oscillation and body compliance help snakes and snake robots stably traverse large, smooth obstaclescited by 1
  7. 2024Offline Goal-Conditioned Reinforcement Learning for Shape Control of Deformable Linear Objectscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Action-Quantized Offline Reinforcement Learning for Robotic Skill Learning
  2. 2025RobustVLA: Robustness-Aware Reinforcement Post-Training for Vision-Language-Action Models
  3. 2021IQ-Learn: Inverse soft-Q Learning for Imitation
  4. 2020Overcoming Model Bias for Robust Offline Deep Reinforcement Learning
  5. 2023Towards Robust Offline Reinforcement Learning under Diverse Data Corruption
  6. 2023Q-Transformer: Scalable Offline Reinforcement Learning via Autoregressive Q-Functions
  7. 2025Uncertainty-Aware Robotic World Model Makes Offline Model-Based Reinforcement Learning Work on Real Robots
  8. 2025PIQL: Projective Implicit Q-Learning with Support Constraint for Offline Reinforcement Learning