# Cross apprenticeship learning framework: Properties and solution approaches

arXiv: https://arxiv.org/abs/2209.02424 · HTML: https://mlsanity.com/paper/2209.02424/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 28 references that are in the map.
- First posted on arXiv 6 Sep 2022.

## Builds on (most cited first, 13 of 13)

- [An Algorithmic Perspective on Imitation Learning](https://mlsanity.com/paper/1811.06711/) (2018)
- [Variational Policy Gradient Method for Reinforcement Learning with General Utilities](https://mlsanity.com/paper/2007.02151/) (2020)
- [On the Sample Complexity of Stability Constrained Imitation Learning](https://mlsanity.com/paper/2102.09161/) (2021)
- [Imitation Learning with Stability and Safety Guarantees](https://mlsanity.com/paper/2012.09293/) (2020)
- [Robust Imitation Learning against Variations in Environment Dynamics](https://mlsanity.com/paper/2206.09314/) (2022)
- [Inverse Reinforcement Learning in Contextual MDPs](https://mlsanity.com/paper/1905.09710/) (2019)
- [Efficient Performance Bounds for Primal-Dual Reinforcement Learning from Demonstrations](https://mlsanity.com/paper/2112.14004/) (2021)
- [Distributed Optimization for Smart Cyber-Physical Networks](https://mlsanity.com/paper/1906.10760/) (2019)
- [Multi-task Reinforcement Learning in Reproducing Kernel Hilbert Spaces via Cross-learning](https://mlsanity.com/paper/2008.11895/) (2020)
- [Inverse Reinforcement Learning: A Control Lyapunov Approach](https://mlsanity.com/paper/2104.04483/) (2021)
- [Large-Scale Markov Decision Problems via the Linear Programming Dual](https://mlsanity.com/paper/1901.01992/) (2019)
- [On the Benefits of Inducing Local Lipschitzness for Robust Generative Adversarial Imitation Learning](https://mlsanity.com/paper/2107.00116/) (2021)
- [Stochastic convex optimization for provably efficient apprenticeship learning](https://mlsanity.com/paper/2201.00039/) (2021)

## Abstract (arXiv)

Apprenticeship learning is a framework in which an agent learns a policy to perform a given task in an environment using example trajectories provided by an expert. In the real world, one might have access to expert trajectories in different environments where the system dynamics is different while the learning task is the same. For such scenarios, two types of learning objectives can be defined. One where the learned policy performs very well in one specific environment and another when it performs well across all environments. To balance these two objectives in a principled way, our work presents the cross apprenticeship learning (CAL) framework. This consists of an optimization problem where an optimal policy for each environment is sought while ensuring that all policies remain close to each other. This nearness is facilitated by one tuning parameter in the optimization problem. We derive properties of the optimizers of the problem as the tuning parameter varies. Since the problem is nonconvex, we provide a convex outer approximation. Finally, we demonstrate the attributes of our framework in the context of a navigation task in a windy gridworld environment.