mlsanityData to 14 Oct 2026

CubeTR: Learning to Solve The Rubiks Cube Using Transformers

arXiv 2111.06036 ↗First posted 11 Nov 2021cs.LG cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Since its first appearance, transformers have been successfully used in wide ranging domains from computer vision to natural language processing. Application of transformers in Reinforcement Learning by reformulating it as a sequence modelling problem was proposed only recently. Compared to other commonly explored reinforcement learning problems, the Rubiks cube poses a unique set of challenges. The Rubiks cube has a single solved state for quintillions of possible configurations which leads to extremely sparse rewards. The proposed model CubeTR attends to longer sequences of actions and addresses the problem of sparse rewards. CubeTR learns how to solve the Rubiks cube from arbitrary starting states without any human prior, and after move regularisation, the lengths of solutions generated by it are expected to be very close to those given by algorithms used by expert human solvers. CubeTR provides insights to the generalisability of learning algorithms to higher dimensional cubes and the applicability of transformers in other relevant sparse reward scenarios.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 26 of its 52 references that are in the map. First posted on arXiv 11 Nov 2021.

Builds on

26 of 52 references.

References · 26

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  5. 2015Trust Region Policy Optimizationcited by 2,580
  6. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  7. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  8. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  9. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  10. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  11. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 786
  12. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 785
14 more
  1. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 734
  2. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 465
  3. 2020End-to-End Video Instance Segmentation with Transformerscited by 443
  4. 2017Deep Reinforcement Learning: An Overviewcited by 333
  5. 2020GPT-GNN: Generative Pre-Training of Graph Neural Networkscited by 303
  6. 2018Large-Scale Study of Curiosity-Driven Learningcited by 280
  7. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 259
  8. 2018Learning by Playing - Solving Sparse Reward Tasks from Scratchcited by 167
  9. 2015Multi-armed Bandit Models for the Optimal Design of Clinical Trials: Benefits and Challengescited by 114
  10. 2019Keeping Your Distance: Solving Sparse Reward Tasks Using Self-Balancing Shaped Rewardscited by 78
  11. 2020DARE: Data Augmented Relation Extraction with GPT-2cited by 43
  12. 2020Abstract Value Iteration for Hierarchical Reinforcement Learningcited by 12
  13. 2018Computational Theories of Curiosity-Driven Learningcited by 11
  14. 2016Neurohex: A Deep Q-learning Hex Agentcited by 5

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025CayleyPy RL: Pathfinding and Reinforcement Learning on Cayley Graphscited by 6
  2. 2025CayleyPy Growth: Efficient growth computations and hundreds of new conjectures on Cayley graphs (Brief version)cited by 1