mlsanityData to 14 Oct 2026

Hadamax Encoding: Elevating Performance in Model-Free Atari

arXiv 2505.15345 ↗First posted 21 May 2025cs.LG cs.AIcited by 7

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Neural network architectures have a large impact in machine learning. In reinforcement learning, network architectures have remained notably simple, as changes often lead to small gains in performance. This work introduces a novel encoder architecture for pixel-based model-free reinforcement learning. The Hadamax (Hadamard max-pooling) encoder achieves state-of-the-art performance by max-pooling Hadamard products between GELU-activated parallel hidden layers. Based on the recent PQN algorithm, the Hadamax encoder achieves state-of-the-art model-free performance in the Atari-57 benchmark. Specifically, without applying any algorithmic hyperparameter modifications, Hadamax-PQN achieves an 80% performance gain over vanilla PQN and significantly surpasses Rainbow-DQN. For reproducibility, the full code is available on hrefhttps://github.com/Jacobkooi/HadamaxGitHub.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 48 of its 61 references that are in the map. First posted on arXiv 21 May 2025.

Builds on

48 of 61 references.

Closest ancestors · 8

Same direction, newest first

  1. 2024SimBa: Simplicity Bias for Scaling Up Parameters in Deep Reinforcement Learning13 Oct
  2. 2024Simplifying Deep Temporal Difference Learning5 Jul
  3. 2024Learning to Play Atari in a World of Tokens3 Jun
  4. 2024Bigger, Regularized, Optimistic: scaling for compute and sample-efficient continuous control25 May
  5. 2024EfficientZero V2: Mastering Discrete and Continuous Control with Limited Data1 Mar
  6. 2024In value-based deep reinforcement learning, a pruned network is a good network19 Feb
  7. 2024Mixtures of Experts Unlock Parameter Scaling for Deep RL13 Feb
  8. 2024Bridging State and History Representations: Understanding Self-Predictive RL17 Jan

Other references · 40

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2017Proximal Policy Optimization Algorithmscited by 15,215
  5. 2016Layer Normalizationcited by 5,669
  6. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  7. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  8. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  9. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  10. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  11. 2023Mastering Diverse Domains through World Modelscited by 1,253
  12. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
28 more
  1. 2020Mastering Atari with Discrete World Modelscited by 1,007
  2. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
  3. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 786
  4. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 785
  5. 2017A Distributional Perspective on Reinforcement Learningcited by 765
  6. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  7. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 407
  8. 2022Temporal Difference Learning for Model Predictive Controlcited by 397
  9. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 318
  10. 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 300
  11. 2021Randomized Ensembled Double Q-Learning: Learning Fast Without a Modelcited by 288
  12. 2017Noisy Networks for Explorationcited by 266
  13. 2021Mastering Atari Games with Limited Datacited by 237
  14. 2018An Introduction to Deep Reinforcement Learningcited by 206
  15. 2016ViZDoom: A Doom-based AI Research Platform for Visual Reinforcement Learningcited by 179
  16. 2023The Dormant Neuron Phenomenon in Deep Reinforcement Learningcited by 174
  17. 2015Training Very Deep Networkscited by 173
  18. 2023Bigger, Better, Faster: Human-level Atari with human-level efficiencycited by 147
  19. 2020Implicit Under-Parameterization Inhibits Data-Efficient Deep Reinforcement Learningcited by 134
  20. 2022Discovered Policy Optimisationcited by 104
  21. 2019CrossQ: Batch Normalization in Deep Reinforcement Learning for Greater Sample Efficiency and Simplicitycited by 95
  22. 2020Sample Factory: Egocentric 3D Control from Pixels at 100000 FPS with Asynchronous Reinforcement Learningcited by 93
  23. 2023STORM: Efficient Stochastic Transformer based World Models for Reinforcement Learningcited by 89
  24. 2022EnvPool: A Highly Parallel Reinforcement Learning Environment Execution Enginecited by 62
  25. 2022Atari-5: Distilling the Arcade Learning Environment down to Five Gamescited by 44
  26. 2021Manifold Regularized Dynamic Network Pruningcited by 43
  27. 2022An Empirical Study of Implicit Regularization in Deep Offline RLcited by 24
  28. 2025Hadamard product in deep learning: Introduction, Advances and Challengescited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 5.

7 papers.

  1. 2025Simplicial Embeddings Improve Sample Efficiency in Actor-Critic Agentscited by 12
  2. 2025Mind the GAP! The Challenges of Scale in Pixel-based Deep Reinforcement Learningcited by 4
  3. 2026Modularized Reinforcement Learning on LLMs: From MDP Creation to Exploration and Learningcited by 2
  4. 2026Higher Resolution, Better Generalization: Unlocking Visual Scaling in Deep Reinforcement Learningcited by 0
  5. 2026Representation Learning Enables Scalable Multitask Deep Reinforcement Learningcited by 0
  6. 2026A Progressive Design Study of Visual Encoders and Value Estimation for Replay-Free Parallelized Q-Learningcited by 0
  7. 2026V-Simba: Unleashing the Architectural Potential of RL in Visual Continuous Controlcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Hadamard Representation: Scaffolding Performance Across Model-free RL
  2. 2024Transforming Game Play: A Comparative Study of DCQN and DTQN Architectures in Reinforcement Learning
  3. 2024Beyond The Rainbow: High Performance Deep Reinforcement Learning on a Desktop PC
  4. 2025Impoola: The Power of Average Pooling for Image-Based Deep Reinforcement Learning
  5. 2024Stable Hadamard Memory: Revitalizing Memory-Augmented Agents for Reinforcement Learning
  6. 2024A quantum-classical reinforcement learning model to play Atari games
  7. 2026On BatchNorm Forward Modes in Value-Based Reinforcement Learning
  8. 2020Smaller World Models for Reinforcement Learning