# Learning to Learn: Meta-Critic Networks for Sample Efficient Learning

arXiv: https://arxiv.org/abs/1706.09529 · HTML: https://mlsanity.com/paper/1706.09529/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Jun 2017.

## Built on by (most cited first, 28 of 28)

- [Meta-learning PINN loss functions](https://mlsanity.com/paper/2107.05544/) (2021)
- [Meta-Learning with Task-Adaptive Loss Function for Few-Shot Learning](https://mlsanity.com/paper/2110.03909/) (2021)
- [Recomposing the Reinforcement Learning Building Blocks with Hypernetworks](https://mlsanity.com/paper/2106.06842/) (2021)
- [Watch, Try, Learn: Meta-Learning from Demonstrations and Reward](https://mlsanity.com/paper/1906.03352/) (2019)
- [On the Effectiveness of Fine-tuning Versus Meta-reinforcement Learning](https://mlsanity.com/paper/2206.03271/) (2022)
- [CoMPS: Continual Meta Policy Search](https://mlsanity.com/paper/2112.04467/) (2021)
- [Efficient Symbolic Policy Learning with Differentiable Symbolic Expression](https://mlsanity.com/paper/2311.02104/) (2023)
- [Learning an Explicit Hyperparameter Prediction Function Conditioned on Tasks](https://mlsanity.com/paper/2107.02378/) (2021)
- [Bitwidth-Adaptive Quantization-Aware Neural Network Training: A Meta-Learning Approach](https://mlsanity.com/paper/2207.10188/) (2022)
- [Hindsight Task Relabelling: Experience Replay for Sparse Reward Meta-RL](https://mlsanity.com/paper/2112.00901/) (2021)
- [Episodic Multi-Task Learning with Heterogeneous Neural Processes](https://mlsanity.com/paper/2310.18713/) (2023)
- [Simple Emergent Action Representations from Multi-Task Policy Training](https://mlsanity.com/paper/2210.09566/) (2022)
- [Learning Action Translator for Meta Reinforcement Learning on Sparse-Reward Tasks](https://mlsanity.com/paper/2207.09071/) (2022)
- [Learning to Plan Before Answering: Self-Teaching LLMs to Learn Abstract Plans for Problem Solving](https://mlsanity.com/paper/2505.00031/) (2025)
- [Adapting to Dynamic LEO-B5G Systems: Meta-Critic Learning Based Efficient Resource Scheduling](https://mlsanity.com/paper/2110.06787/) (2021)
- [Meta-Reinforcement Learning for Heuristic Planning](https://mlsanity.com/paper/2107.02603/) (2021)
- [Multi-Domain Learning by Meta-Learning: Taking Optimal Steps in Multi-Domain Loss Landscapes by Inner-Loop Learning](https://mlsanity.com/paper/2102.13147/) (2021)
- [Human-Inspired Framework to Accelerate Reinforcement Learning](https://mlsanity.com/paper/2303.08115/) (2023)
- [Black Box Meta-Learning Intrinsic Rewards](https://mlsanity.com/paper/2407.21546/) (2024)
- [Relation-Aware Meta-Learning for Zero-shot Sketch-Based Image Retrieval](https://mlsanity.com/paper/2412.00120/) (2024)
- [Combining Bayesian Inference and Reinforcement Learning for Agent Decision Making: A Review](https://mlsanity.com/paper/2505.07911/) (2025)
- [Model Based Meta Learning of Critics for Policy Gradients](https://mlsanity.com/paper/2204.02210/) (2022)
- [Implicit Training of Energy Model for Structure Prediction](https://mlsanity.com/paper/2211.11649/) (2022)
- [Acceleration in Policy Optimization](https://mlsanity.com/paper/2306.10587/) (2023)
- [RL3: Boosting Meta Reinforcement Learning via RL inside RL2](https://mlsanity.com/paper/2306.15909/) (2023)
- [Hypothesis Network Planned Exploration for Rapid Meta-Reinforcement Learning Adaptation](https://mlsanity.com/paper/2311.03701/) (2023)
- [Adaptive Policy Backbone via Shared Network](https://mlsanity.com/paper/2509.22310/) (2025)
- [Meta-RL with Shared Representations Enables Fast Adaptation in Energy Systems](https://mlsanity.com/paper/2603.08418/) (2026)

## Abstract (arXiv)

We propose a novel and flexible approach to meta-learning for learning-to-learn from only a few examples. Our framework is motivated by actor-critic reinforcement learning, but can be applied to both reinforcement and supervised learning. The key idea is to learn a meta-critic: an action-value function neural network that learns to criticise any actor trying to solve any specified task. For supervised learning, this corresponds to the novel idea of a trainable task-parametrised loss generator. This meta-critic approach provides a route to knowledge transfer that can flexibly deal with few-shot and semi-supervised conditions for both reinforcement and supervised learning. Promising results are shown on both reinforcement and supervised learning problems.