Data to 7 Oct 2026

CrossQ: Batch Normalization in Deep Reinforcement Learning for Greater Sample Efficiency and Simplicity

arXiv 1902.05605 ↗First posted 14 Feb 2019cs.LG stat.MLcited by 89

Not on the map. Why?

Abstract (arXiv)

Sample efficiency is a crucial problem in deep reinforcement learning. Recent algorithms, such as REDQ and DroQ, found a way to improve the sample efficiency by increasing the update-to-data (UTD) ratio to 20 gradient update steps on the critic per environment sample. However, this comes at the expense of a greatly increased computational cost. To reduce this computational burden, we introduce CrossQ: A lightweight algorithm for continuous control tasks that makes careful use of Batch Normalization and removes target networks to surpass the current state-of-the-art in sample efficiency while maintaining a low UTD ratio of 1. Notably, CrossQ does not rely on advanced bias-reduction schemes used in current methods. CrossQ's contributions are threefold: (1) it matches or surpasses current state-of-the-art methods in terms of sample efficiency, (2) it substantially reduces the computational cost compared to REDQ and DroQ, (3) it is easy to implement, requiring just a few lines of code on top of SAC.

Key facts. Cited by 89 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Feb 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 2; 2023: 1; 2024: 16; 2025: 36; 2026: 31.

89 papers.

Sort:

By area:

2024Bigger, Regularized, Optimistic: scaling for compute and sample-efficient continuous controlcited by 104
2024Simplifying Deep Temporal Difference Learningcited by 71
2024SimBa: Simplicity Bias for Scaling Up Parameters in Deep Reinforcement Learningcited by 70
2025DIME:Diffusion-Based Maximum Entropy Reinforcement Learningcited by 52
2025Latent Action Learning Requires Supervision in the Presence of Distractorscited by 40
2025Hyperspherical Normalization for Scalable Deep Reinforcement Learningcited by 35
2025Maximum Entropy Reinforcement Learning with Diffusion Policycited by 29
2024Plasticity Loss in Deep Reinforcement Learning: A Surveycited by 24
2025Bigger, Regularized, Categorical: High-Capacity Value Functions are Efficient Multi-Task Learnerscited by 23
2025floq: Training Critics via Flow-Matching for Scaling Compute in Value-Based RLcited by 20
2025Scaling Off-Policy Reinforcement Learning with Batch and Weight Normalizationcited by 19
2024Interpretable and Editable Programmatic Tree Policies for Reinforcement Learningcited by 18

38 more

2024Dissecting Deep RL with High Update Ratios: Combatting Value Divergencecited by 17
2025XQC: Well-conditioned Optimization Accelerates Deep Reinforcement Learningcited by 17
2025Chunking the Critic: A Transformer-based Soft Actor-Critic with N-Step Returnscited by 16
2024BlendRL: A Framework for Merging Symbolic and Neural Policy Learningcited by 14
2024Stabilizing Reinforcement Learning in Differentiable Multiphysics Simulationcited by 14
2022A System for Morphology-Task Generalization via Unified Representation and Behavior Distillationcited by 12
2025Network Sparsity Unlocks the Scaling Potential of Deep Reinforcement Learningcited by 11
2021Convergent and Efficient Deep Q Network Algorithmcited by 8
2025Gait in Eight: Efficient On-Robot Learning for Omnidirectional Quadruped Locomotioncited by 7
2023On the Theory of Risk-Aware Agents: Bridging Actor-Critic and Economicscited by 6
2024Iterated Q-Network: Beyond One-Step Bellman Updates in Deep Reinforcement Learningcited by 6
2025Eau De Q-Network: Adaptive Distillation of Neural Networks in Deep Reinforcement Learningcited by 6
2025Hadamax Encoding: Elevating Performance in Model-Free Ataricited by 6
2021Adaptively Calibrated Critic Estimates for Deep Reinforcement Learningcited by 5
2026FlashSAC: Fast and Stable Off-Policy Reinforcement Learning for High-Dimensional Robot Controlcited by 5
2025Bridging the Performance Gap Between Target-Free and Target-Based Reinforcement Learningcited by 4
2026Scalable Exploration for High-Dimensional Continuous Control via Value-Guided Flowcited by 4
2022Batch Layer Normalization, A new normalization layer for CNNs and RNNcited by 3
2024Adaptive Q-Network: On-the-fly Target Selection for Deep Reinforcement Learningcited by 3
2024Model-free reinforcement learning with noisy actions for automated experimental control in opticscited by 2
2024Offline-to-online Reinforcement Learning for Image-based Grasping with Scarce Demonstrationscited by 2
2025Constructing an Optimal Behavior Basis for the Option Keyboardcited by 2
2025Tactile MNIST: Benchmarking Active Tactile Perceptioncited by 2
2025ASVSim (AirSim for Surface Vehicles): A High-Fidelity Simulation Framework for Autonomous Surface Vehicle Researchcited by 2
2025Sparsity-Driven Plasticity in Multi-Task Reinforcement Learningcited by 2
2025Use the Online Network If You Can: Towards Fast and Stable Reinforcement Learningcited by 2
2025Towards Safe Maneuvering of Double-Ackermann-Steering Robots with a Soft Actor-Critic Frameworkcited by 2
2026A Review of Online Diffusion Policy RL Algorithms for Scalable Robotic Controlcited by 2
2026Temporal Difference Learning with Constrained Initial Representationscited by 2
2026Diffusing to Coordinate: Efficient Online Multi-Agent Diffusion Policiescited by 2
2026Truncated Rectified Flow Policy for Reinforcement Learning with One-Step Samplingcited by 2
2026FLAG: Flow Policy MaxEnt-RL by Latent Augmented Guidancecited by 2
2024Efficient Off-Policy Learning for High-Dimensional Action Spacescited by 1
2024A Differential and Pointwise Control Approach to Reinforcement Learningcited by 1
2024Hadamard Representation: Scaffolding Performance Across Model-free RLcited by 1
2025Circular Microalgae-Based Carbon Control for Net Zerocited by 1
2025Massively Scaling Explicit Policy-conditioned Value Functionscited by 1
2025Apple: Toward General Active Perception via Reinforcement Learningcited by 1

50 most cited of 89 All 89 (JSON)