# On Inductive Biases in Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/1907.02908 · HTML: https://mlsanity.com/paper/1907.02908/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jul 2019.

## Built on by (most cited first, 24 of 24)

- [Automated Reinforcement Learning (AutoRL): A Survey and Open Problems](https://mlsanity.com/paper/2201.03916/) (2022)
- [Muesli: Combining Improvements in Policy Optimization](https://mlsanity.com/paper/2104.06159/) (2021)
- [On the consistency of hyper-parameter selection in value-based deep reinforcement learning](https://mlsanity.com/paper/2406.17523/) (2024)
- [Compositional Multi-Object Reinforcement Learning with Linear Relation Networks](https://mlsanity.com/paper/2201.13388/) (2022)
- [Causal Multi-Agent Reinforcement Learning: Review and Open Problems](https://mlsanity.com/paper/2111.06721/) (2021)
- [Transformers in Reinforcement Learning: A Survey](https://mlsanity.com/paper/2307.05979/) (2023)
- [Towards Automatic Actor-Critic Solutions to Continuous Control](https://mlsanity.com/paper/2106.08918/) (2021)
- [In-Context Reinforcement Learning via Communicative World Models](https://mlsanity.com/paper/2508.06659/) (2025)
- [Selective Credit Assignment](https://mlsanity.com/paper/2202.09699/) (2022)
- [Reinforcement Learning with Physics-Informed Symbolic Program Priors for Zero-Shot Wireless Indoor Navigation](https://mlsanity.com/paper/2506.22365/) (2025)
- [SoK: The Pitfalls of Deep Reinforcement Learning for Cybersecurity](https://mlsanity.com/paper/2602.08690/) (2026)
- [Speeding up deep neural network-based planning of local car maneuvers via efficient B-spline path construction](https://mlsanity.com/paper/2203.06963/) (2022)
- [Multi-task Batch Reinforcement Learning with Metric Learning](https://mlsanity.com/paper/1909.11373/) (2019)
- [Evaluating the progress of Deep Reinforcement Learning in the real world: aligning domain-agnostic and domain-specific research](https://mlsanity.com/paper/2107.03015/) (2021)
- [Evolving Pareto-Optimal Actor-Critic Algorithms for Generalizability and Stability](https://mlsanity.com/paper/2204.04292/) (2022)
- [Temporally Extended Successor Representations](https://mlsanity.com/paper/2209.12331/) (2022)
- [Differentiable Tree Search Network](https://mlsanity.com/paper/2401.11660/) (2024)
- [EvoRL: A GPU-accelerated Framework for Evolutionary Reinforcement Learning](https://mlsanity.com/paper/2501.15129/) (2025)
- [Utilizing Skipped Frames in Action Repeats via Pseudo-Actions](https://mlsanity.com/paper/2105.03041/) (2021)
- [Loss Dynamics of Temporal Difference Reinforcement Learning](https://mlsanity.com/paper/2307.04841/) (2023)
- [Hierarchical Reinforcement Learning for Neural Network Compression (HiReLC): Pruning and Quantization](https://mlsanity.com/paper/2606.26002/) (2026)
- [The Mechanism Matters: When Knowledge Graphs Help Reinforcement Learning](https://mlsanity.com/paper/2607.19616/) (2026)
- [Adaptive Multi-Horizon Reinforcement Learning](https://mlsanity.com/paper/2607.20656/) (2026)
- [Independent Multi-Agent Reinforcement Learning with Counterfactual Semantic-Social World Models](https://mlsanity.com/paper/2610.07704/) (2026)

## Abstract (arXiv)

Many deep reinforcement learning algorithms contain inductive biases that sculpt the agent's objective and its interface to the environment. These inductive biases can take many forms, including domain knowledge and pretuned hyper-parameters. In general, there is a trade-off between generality and performance when algorithms use such biases. Stronger biases can lead to faster learning, but weaker biases can potentially lead to more general algorithms. This trade-off is important because inductive biases are not free; substantial effort may be required to obtain relevant domain knowledge or to tune hyper-parameters effectively. In this paper, we re-examine several domain-specific components that bias the objective and the environmental interface of common deep reinforcement learning agents. We investigated whether the performance deteriorates when these components are replaced with adaptive solutions from the literature. In our experiments, performance sometimes decreased with the adaptive components, as one might expect when comparing to components crafted for the domain, but sometimes the adaptive components performed better. We investigated the main benefit of having fewer domain-specific components, by comparing the learning performance of the two systems on a different set of continuous control problems, without additional tuning of either system. As hypothesized, the system with adaptive components performed better on many of the new tasks.