# Generalization to New Actions in Reinforcement Learning

arXiv: https://arxiv.org/abs/2011.01928 · HTML: https://mlsanity.com/paper/2011.01928/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Nov 2020.

## Built on by (most cited first, 25 of 25)

- [A Survey of Zero-shot Generalisation in Deep Reinforcement Learning](https://mlsanity.com/paper/2111.09794/) (2021)
- [Augmenting Reinforcement Learning with Behavior Primitives for Diverse Manipulation Tasks](https://mlsanity.com/paper/2110.03655/) (2021)
- [Learning to Synthesize Programs as Interpretable and Generalizable Policies](https://mlsanity.com/paper/2108.13643/) (2021)
- [Grounding Multimodal Large Language Models in Actions](https://mlsanity.com/paper/2406.07904/) (2024)
- [Active Evaluation Acquisition for Efficient LLM Benchmarking](https://mlsanity.com/paper/2410.05952/) (2024)
- [In-Context Reinforcement Learning for Variable Action Spaces](https://mlsanity.com/paper/2312.13327/) (2023)
- [I-PHYRE: Interactive Physical Reasoning](https://mlsanity.com/paper/2312.03009/) (2023)
- [Learn the Time to Learn: Replay Scheduling in Continual Learning](https://mlsanity.com/paper/2209.08660/) (2022)
- [SPARTAN: A Sparse Transformer World Model Attending to What Matters](https://mlsanity.com/paper/2411.06890/) (2024)
- [A Survey Analyzing Generalization in Deep Reinforcement Learning](https://mlsanity.com/paper/2401.02349/) (2024)
- [Creative Problem Solving in Artificially Intelligent Agents: A Survey and Framework](https://mlsanity.com/paper/2204.10358/) (2022)
- [Dynamic Neighborhood Construction for Structured Large Discrete Action Spaces](https://mlsanity.com/paper/2305.19891/) (2023)
- [BusyBot: Learning to Interact, Reason, and Plan in a BusyBoard Environment](https://mlsanity.com/paper/2207.08192/) (2022)
- [Action Pick-up in Dynamic Action Space Reinforcement Learning](https://mlsanity.com/paper/2304.00873/) (2023)
- [The Path To Autonomous Cyber Defense](https://mlsanity.com/paper/2404.10788/) (2024)
- [Mitigating Suboptimality of Deterministic Policy Gradients in Complex Q-functions](https://mlsanity.com/paper/2410.11833/) (2024)
- [Neural Force Field: Few-shot Learning of Generalized Physical Reasoning](https://mlsanity.com/paper/2502.08987/) (2025)
- [Attacks and Defenses Against LLM Fingerprinting](https://mlsanity.com/paper/2508.09021/) (2025)
- [Learning to reason about and to act on physical cascading events](https://mlsanity.com/paper/2202.01108/) (2022)
- [AIGenC: An AI generalisation model via creativity](https://mlsanity.com/paper/2205.09738/) (2022)
- [DCT: Dual Channel Training of Action Embeddings for Reinforcement Learning with Large Discrete Action Spaces](https://mlsanity.com/paper/2306.15913/) (2023)
- [Zero-Shot Action Generalization with Limited Observations](https://mlsanity.com/paper/2503.08867/) (2025)
- [A representational framework for learning and encoding structurally enriched trajectories in complex agent environments](https://mlsanity.com/paper/2503.13194/) (2025)
- [Actor-Free Continuous Control via Structurally Maximizable Q-Functions](https://mlsanity.com/paper/2510.18828/) (2025)
- [Training Language Models to Cooperate with Inference-Time Controllers](https://mlsanity.com/paper/2607.23771/) (2026)

## Abstract (arXiv)

A fundamental trait of intelligence is the ability to achieve goals in the face of novel circumstances, such as making decisions from new action choices. However, standard reinforcement learning assumes a fixed set of actions and requires expensive retraining when given a new action set. To make learning agents more adaptable, we introduce the problem of zero-shot generalization to new actions. We propose a two-stage framework where the agent first infers action representations from action information acquired separately from the task. A policy flexible to varying action sets is then trained with generalization objectives. We benchmark generalization on sequential tasks, such as selecting from an unseen tool-set to solve physical reasoning puzzles and stacking towers with novel 3D shapes. Videos and code are available at https://sites.google.com/view/action-generalization