MePoly: Max Entropy Polynomial Policy Optimization
On the map: Offline RL and imitation learning from data → Inverse reinforcement learning
Abstract (arXiv)
Stochastic Optimal Control provides a unified mathematical framework for solving complex decision-making problems, encompassing paradigms such as maximum entropy reinforcement learning(RL) and imitation learning(IL). However, conventional parametric policies often struggle to represent the multi-modality of the solutions. Though diffusion-based policies are aimed at recovering the multi-modality, they lack an explicit probability density, which complicates policy-gradient optimization. To bridge this gap, we propose MePoly, a novel policy parameterization based on polynomial energy-based models. MePoly provides an explicit, tractable probability density, enabling exact entropy maximization. Theoretically, we ground our method in the classical moment problem, leveraging the universal approximation capabilities for arbitrary distributions. Empirically, we demonstrate that MePoly effectively captures complex non-convex manifolds and outperforms baselines in performance across diverse benchmarks.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 19 of its 35 references that are in the map. First posted on arXiv 19 Feb 2026.
Builds on
19 of 35 references.
References · 19
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
- 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
- 2015Trust Region Policy Optimizationcited by 2,503
- 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
- 2017Reinforcement Learning with Deep Energy-Based Policiescited by 659
- 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learningcited by 543
- 2021Implicit Behavioral Cloningcited by 442
- 2024Diffusion Policy Policy Optimizationcited by 255
- 2021ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrationscited by 193
- 2015Taming the Noise in Reinforcement Learning via Soft Updatescited by 125
- 2025Flow Matching Policy Gradientscited by 86
7 more
- 2019Discretizing Continuous Action Space for On-Policy Optimizationcited by 65
- 2024Learning Multimodal Behaviors from Scratch with Diffusion Policy Gradientcited by 49
- 2021Probabilistic Mixture-of-Experts for Efficient Deep Reinforcement Learningcited by 26
- 2023Reparameterized Policy Learning for Multimodal Trajectory Optimizationcited by 20
- 2021Learning Latent Actions to Control Assistive Robotscited by 17
- 2025Action Chunking and Exploratory Data Collection Yield Exponential Improvements in Behavior Cloning for Continuous Controlcited by 15
- 2025Max Entropy Moment Kalman Filter for Polynomial Systems with Arbitrary Noisecited by 4
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025DIME:Diffusion-Based Maximum Entropy Reinforcement Learning
- 2025Diffusion-Augmented Markov Decision Processes for Maximum Entropy Reinforcement Learning
- 2026Scalable Maximum Entropy Reinforcement Learning for Diffusion Policies via Adjoint Matching
- 2026Boosting Maximum Entropy Reinforcement Learning via One-Step Flow Matching
- 2022Weighted Maximum Entropy Inverse Reinforcement Learning
- 2020Revisiting Maximum Entropy Inverse Reinforcement Learning: New Perspectives and Algorithms
- 2026Entropy Polarity in Reinforcement Fine-Tuning: Direction, Asymmetry, and Control
- 2025Failure Modes of Maximum Entropy RLHF