GIPO: Gaussian Importance Sampling Policy Optimization
On the map: Deep reinforcement learning methods → RL theory and sample complexity
Abstract (arXiv)
Post-training with reinforcement learning (RL) has recently shown strong promise for advancing multimodal agents beyond supervised imitation. However, RL remains limited by poor data efficiency, particularly in settings where interaction data are scarce and quickly become outdated. To address this challenge, GIPO (Gaussian Importance sampling Policy Optimization) is proposed as a policy optimization objective based on truncated importance sampling, replacing hard clipping with a log-ratio-based Gaussian trust weight to softly damp extreme importance ratios while maintaining non-zero gradients. Theoretical analysis shows that GIPO introduces an implicit, tunable constraint on the update magnitude, while concentration bounds guarantee robustness and stability under finite-sample estimation. Experimental results show that GIPO achieves state-of-the-art performance among clipping-based baselines across a wide range of replay buffer sizes, from near on-policy to highly stale data, while exhibiting superior bias--variance trade-off, high training stability and improved sample efficiency. Code is available at https://github.com/distanceLu/GIPO.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 18 of its 26 references that are in the map. First posted on arXiv 4 Mar 2026.
Builds on
18 of 26 references.
Closest ancestors · 2
Same direction, newest first
- 2025Divergence-Augmented Policy Optimization25 Jan
- 2024Reusing Historical Trajectories in Natural Policy Gradient via Importance Sampling: Convergence and Convergence Rate1 Mar
Other references · 16
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
- 2015Trust Region Policy Optimizationcited by 2,580
- 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
- 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
- 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
- 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,241
- 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
- 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
- 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Successcited by 883
- 2016Safe and Efficient Off-Policy Reinforcement Learningcited by 264
- 2016Sample Efficient Actor-Critic with Experience Replaycited by 188
4 more
- 2025Soft Adaptive Policy Optimizationcited by 96
- 2019Truly Proximal Policy Optimizationcited by 69
- 2018Policy Optimization via Importance Samplingcited by 47
- 2020Stable Policy Optimization via Off-Policy Divergence Regularizationcited by 16
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025TIC-GRPO: Provable and Efficient Optimization for Reinforcement Learning from Human Feedback
- 2025Deep Gaussian Process Proximal Policy Optimization
- 2026F-TIS: Harnessing Diverse Models in Collaborative GRPO
- 2025ESPO: Entropy Importance Sampling Policy Optimization
- 2025GRPO-Guard: Mitigating Implicit Over-Optimization in Flow Matching via Regulated Clipping
- 2026How Off-Policy Can GRPO Be? Mu-GRPO for Efficient LLM Reinforcement Learning
- 2026Prompts Live on an Arc: Gaussian Curricula in Fisher--Rao Coordinates for Rollout-Efficient GRPO
- 2026F-GRPO: Don't Let Your Policy Learn the Obvious and Forget the Rare