Data to 7 Oct 2026

Reinforcement Learning via Fenchel-Rockafellar Duality

arXiv 2001.01866 ↗First posted 7 Jan 2020cs.LG stat.MLcited by 100

Not on the map. Why?

Cited by 39 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We review basic concepts of convex duality, focusing on the very general and supremely useful Fenchel-Rockafellar duality. We summarize how this duality may be applied to a variety of reinforcement learning (RL) settings, including policy evaluation or optimization, online or offline learning, and discounted or undiscounted rewards. The derivations yield a number of intriguing results, including the ability to perform policy evaluation and on-policy policy gradient with behavior-agnostic offline data and methods to learn a policy via max-likelihood optimization. Although many of these results have appeared previously in various forms, we provide a unified treatment and perspective on these results, which we hope will enable researchers to better use and apply the tools of convex duality to make further progress in RL.

Key facts. Cited by 100 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Jan 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 3; 2021: 15; 2022: 22; 2023: 20; 2024: 19; 2025: 18; 2026: 3.

100 papers.

Sort:

By area:

2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
2020MOReL : Model-Based Offline Reinforcement Learningcited by 582
2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Trainingcited by 384
2020Is Pessimism Provably Efficient for Offline RL?cited by 362
2021Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimismcited by 275
2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 169
2022Offline Reinforcement Learning with Realizability and Single-policy Concentrabilitycited by 122
2021Provable Benefits of Actor-Critic Methods for Offline Reinforcement Learningcited by 109
2022A Review of Off-Policy Evaluation in Reinforcement Learningcited by 92
2021A Workflow for Offline Model-Free Robotic Reinforcement Learningcited by 68
2022How Far I'll Go: Offline Goal-Conditioned Reinforcement Learning via f-Advantage Regressioncited by 65
2022Making Linear MDPs Practical via Contrastive Representation Learningcited by 49

38 more

2023Dual RL: Unification and New Methods for Reinforcement and Imitation Learningcited by 48
2021Twice regularized MDPs and the equivalence between robustness and regularizationcited by 46
2021Universal Off-Policy Evaluationcited by 43
2022Versatile Offline Imitation from Observations and Examples via Regularized State-Occupancy Matchingcited by 43
2024Diffusion-DICE: In-Sample Diffusion Guidance for Offline Reinforcement Learningcited by 42
2022DARA: Dynamics-Aware Reward Augmentation in Offline Reinforcement Learningcited by 40
2021The Information Geometry of Unsupervised Reinforcement Learningcited by 36
2024Diffusion Actor-Critic: Formulating Constrained Policy Iteration as Diffusion Noise Regression for Offline Reinforcement Learningcited by 33
2022Optimal Conservative Offline RL with General Function Approximation via Augmented Lagrangiancited by 29
2025Think Smarter not Harder: Adaptive Reasoning with Inference Aware Optimizationcited by 29
2021Density Constrained Reinforcement Learningcited by 25
2022Your Policy Regularizer is Secretly an Adversarycited by 21
2023SMORE: Score Models for Offline Goal-Conditioned Reinforcement Learningcited by 21
2024ODICE: Revealing the Mystery of Distribution Correction Estimation via Orthogonal-gradient Updatecited by 21
2022When is Realizability Sufficient for Off-Policy Reinforcement Learning?cited by 17
2023Offline Primal-Dual Reinforcement Learning for Linear MDPscited by 17
2024Proto Successor Measure: Representing the Behavior Space of an RL Agentcited by 17
2023Mind the Gap: Offline Policy Optimization for Imperfect Rewardscited by 15
2024A Distributional Analogue to the Successor Representationcited by 15
2022A Unified Framework for Alternating Offline Model Training and Policy Learningcited by 14
2021Continual Learning In Environments With Polynomial Mixing Timescited by 13
2024Offline Multi-task Transfer RL with Representational Penalizationcited by 13
2022Model-Free and Model-Based Policy Evaluation when Causality is Uncertaincited by 12
2022Bellman Residual Orthogonalization for Offline Reinforcement Learningcited by 10
2023TOM: Learning Policy-Aware Models for Model-Based Reinforcement Learning via Transition Occupancy Matchingcited by 10
2023AlberDICE: Addressing Out-Of-Distribution Joint Actions in Offline Multi-Agent RL via Alternating Stationary Distribution Correction Estimationcited by 10
2023Provable Representation with Efficient Planning for Partial Observable Reinforcement Learningcited by 10
2024A Dual Approach to Imitation Learning from Observations with Offline Datasetscited by 10
2024Near-Optimal Policy Identification in Robust Constrained Markov Decision Processes via Epigraph Formcited by 10
2025Offline Reinforcement Learning in Large State Spaces: Algorithms and Guaranteescited by 10
2021Efficient Performance Bounds for Primal-Dual Reinforcement Learning from Demonstrationscited by 9
2023Offline Minimax Soft-Q-learning Under Realizability and Partial Coveragecited by 9
2023A Unified View on Solving Objective Mismatch in Model-Based Reinforcement Learningcited by 9
2024Learning from Sparse Offline Datasets via Conservative Density Estimationcited by 9
2022Beyond the Return: Off-policy Function Estimation under User-specified Error-measuring Distributionscited by 8
2024ComaDICE: Offline Cooperative Multi-Agent Reinforcement Learning with Stationary Distribution Shift Regularizationcited by 8
2022A Unified Perspective on Value Backup and Exploration in Monte-Carlo Tree Searchcited by 6
2023Debiasing Conditional Stochastic Optimizationcited by 6

50 most cited of 100 All 100 (JSON)