# MePoly: Max Entropy Polynomial Policy Optimization

arXiv: https://arxiv.org/abs/2602.17832 · HTML: https://mlsanity.com/paper/2602.17832/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 19 of its 35 references that are in the map.
- First posted on arXiv 19 Feb 2026.

## Placed in

- Offline RL and imitation learning from data / [Inverse reinforcement learning](https://mlsanity.com/offlinerl/inverse-reinforcement-learning/)
- Deep reinforcement learning methods / [RL with diffusion and flow models](https://mlsanity.com/deeprl/rl-with-diffusion-and-flow-models/)

## Builds on (most cited first, 19 of 19)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware](https://mlsanity.com/paper/2304.13705/) (2023)
- [Reinforcement Learning with Deep Energy-Based Policies](https://mlsanity.com/paper/1702.08165/) (2017)
- [Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learning](https://mlsanity.com/paper/2208.06193/) (2022)
- [Implicit Behavioral Cloning](https://mlsanity.com/paper/2109.00137/) (2021)
- [Diffusion Policy Policy Optimization](https://mlsanity.com/paper/2409.00588/) (2024)
- [ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrations](https://mlsanity.com/paper/2107.14483/) (2021)
- [Taming the Noise in Reinforcement Learning via Soft Updates](https://mlsanity.com/paper/1512.08562/) (2015)
- [Flow Matching Policy Gradients](https://mlsanity.com/paper/2507.21053/) (2025)
- [Discretizing Continuous Action Space for On-Policy Optimization](https://mlsanity.com/paper/1901.10500/) (2019)
- [Learning Multimodal Behaviors from Scratch with Diffusion Policy Gradient](https://mlsanity.com/paper/2406.00681/) (2024)
- [Probabilistic Mixture-of-Experts for Efficient Deep Reinforcement Learning](https://mlsanity.com/paper/2104.09122/) (2021)
- [Reparameterized Policy Learning for Multimodal Trajectory Optimization](https://mlsanity.com/paper/2307.10710/) (2023)
- [Learning Latent Actions to Control Assistive Robots](https://mlsanity.com/paper/2107.02907/) (2021)
- [Action Chunking and Exploratory Data Collection Yield Exponential Improvements in Behavior Cloning for Continuous Control](https://mlsanity.com/paper/2507.09061/) (2025)
- [Max Entropy Moment Kalman Filter for Polynomial Systems with Arbitrary Noise](https://mlsanity.com/paper/2506.00838/) (2025)

## Built on by (most cited first, 3 of 3)

- [Embedding Hybrid Systems into Continuous Latent Vector Fields](https://mlsanity.com/paper/2606.10596/) (2026)
- [The Score Kalman Filter](https://mlsanity.com/paper/2605.16644/) (2026)
- [Learning Continuous Neural Representation of Stochastic Hybrid Systems](https://mlsanity.com/paper/2609.38893/) (2026)

## Abstract (arXiv)

Stochastic Optimal Control provides a unified mathematical framework for solving complex decision-making problems, encompassing paradigms such as maximum entropy reinforcement learning(RL) and imitation learning(IL). However, conventional parametric policies often struggle to represent the multi-modality of the solutions. Though diffusion-based policies are aimed at recovering the multi-modality, they lack an explicit probability density, which complicates policy-gradient optimization. To bridge this gap, we propose MePoly, a novel policy parameterization based on polynomial energy-based models. MePoly provides an explicit, tractable probability density, enabling exact entropy maximization. Theoretically, we ground our method in the classical moment problem, leveraging the universal approximation capabilities for arbitrary distributions. Empirically, we demonstrate that MePoly effectively captures complex non-convex manifolds and outperforms baselines in performance across diverse benchmarks.