# Policy Learning Using Weak Supervision

arXiv: https://arxiv.org/abs/2010.01748 · HTML: https://mlsanity.com/paper/2010.01748/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- First posted on arXiv 5 Oct 2020.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Built on by (most cited first, 13 of 13)

- [Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotations](https://mlsanity.com/paper/2110.12088/) (2021)
- [Detecting Corrupted Labels Without Training a Model to Predict](https://mlsanity.com/paper/2110.06283/) (2021)
- [To Aggregate or Not? Learning with Separate Noisy Labels](https://mlsanity.com/paper/2206.07181/) (2022)
- [Beyond Images: Label Noise Transition Matrix Estimation for Tasks with Lower-Quality Features](https://mlsanity.com/paper/2202.01273/) (2022)
- [Open-Sampling: Exploring Out-of-Distribution data for Re-balancing Long-tailed datasets](https://mlsanity.com/paper/2206.08802/) (2022)
- [Improving Reinforcement Learning from Human Feedback Using Contrastive Rewards](https://mlsanity.com/paper/2403.07708/) (2024)
- [Weak Proxies are Sufficient and Preferable for Fairness with Missing Sensitive Attributes](https://mlsanity.com/paper/2210.03175/) (2022)
- [Adv3D: Generating Safety-Critical 3D Objects through Closed-Loop Simulation](https://mlsanity.com/paper/2311.01446/) (2023)
- [Large Language Models as Agents in Two-Player Games](https://mlsanity.com/paper/2402.08078/) (2024)
- [Learning Discriminative Dynamics with Label Corruption for Noisy Label Detection](https://mlsanity.com/paper/2405.19902/) (2024)
- [Distributional Reward Estimation for Effective Multi-Agent Deep Reinforcement Learning](https://mlsanity.com/paper/2210.07636/) (2022)
- [When VLMs Meet Image Classification: Test Sets Renovation via Missing Label Identification](https://mlsanity.com/paper/2505.16149/) (2025)
- [Learning Complicated Manipulation Skills via Deterministic Policy with Limited Demonstrations](https://mlsanity.com/paper/2303.16469/) (2023)

## Abstract (arXiv)

Most existing policy learning solutions require the learning agents to receive high-quality supervision signals such as well-designed rewards in reinforcement learning (RL) or high-quality expert demonstrations in behavioral cloning (BC). These quality supervisions are usually infeasible or prohibitively expensive to obtain in practice. We aim for a unified framework that leverages the available cheap weak supervisions to perform policy learning efficiently. To handle this problem, we treat the "weak supervision" as imperfect information coming from a peer agent, and evaluate the learning agent's policy based on a "correlated agreement" with the peer agent's policy (instead of simple agreements). Our approach explicitly punishes a policy for overfitting to the weak supervision. In addition to theoretical guarantees, extensive evaluations on tasks including RL with noisy rewards, BC with weak demonstrations, and standard policy co-training show that our method leads to substantial performance improvements, especially when the complexity or the noise of the learning environments is high.