mlsanityData to 7 Oct 2026

Learning to Solve Constraint Satisfaction Problems with Recurrent Transformer

arXiv 2307.04895 ↗First posted 10 Jul 2023cs.AI cs.LGcited by 21

Not on the map. Why?

Abstract (arXiv)

Constraint satisfaction problems (CSPs) are about finding values of variables that satisfy the given constraints. We show that Transformer extended with recurrence is a viable approach to learning to solve CSPs in an end-to-end manner, having clear advantages over state-of-the-art methods such as Graph Neural Networks, SATNet, and some neuro-symbolic models. With the ability of Transformer to handle visual input, the proposed Recurrent Transformer can straightforwardly be applied to visual constraint reasoning problems while successfully addressing the symbol grounding problem. We also show how to leverage deductive knowledge of discrete constraints in the Transformer's inductive learning to achieve sample-efficient learning and semi-supervised learning for CSPs.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 22 of its 39 references that are in the map. First posted on arXiv 10 Jul 2023.

Builds on

22 of 39 references.

References · 22

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  4. 2017Graph Attention Networkscited by 8,854
  5. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  6. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,100
  7. 2020A Generalization of Transformer Networks to Graphscited by 653
  8. 2020Heterogeneous Graph Transformercited by 581
  9. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 571
  10. 2019Graph Transformer Networkscited by 520
  11. 2020Multi-modal Transformer for Video Retrievalcited by 395
  12. 2019Neural Probabilistic Logic Programming in DeepProbLogcited by 389
10 more
  1. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoningcited by 331
  2. 2017A Semantic Loss Function for Deep Learning with Symbolic Knowledgecited by 267
  3. 2019SATNet: Bridging deep learning and logical reasoning using a differentiable satisfiability solvercited by 173
  4. 2021Improving Coherence and Consistency in Neural Sequence Models with Dual-System, Neuro-Symbolic Reasoningcited by 85
  5. 2019Graph Transformer for Graph-to-Sequence Learningcited by 82
  6. 2017Recurrent Relational Networkscited by 71
  7. 2020Neural-Symbolic Integration: A Compositional Perspectivecited by 45
  8. 2019Modeling Recurrence for Transformercited by 26
  9. 2021Techniques for Symbol Grounding with SATNetcited by 24
  10. 2019Deep Reasoning Networks: Thinking Fast and Slowcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 4; 2025: 7; 2026: 9.

21 papers.

Sort:

By area:

  1. 2024Artificial Kuramoto Oscillatory Neuronscited by 37
  2. 2024Learning Iterative Reasoning through Energy Diffusioncited by 36
  3. 2024On Expressive Power of Looped Transformers: Theoretical Analysis and Enhancement via Timestep Encodingcited by 28
  4. 2024Neuro-symbolic Learning Yielding Logical Constraintscited by 12
  5. 2025Self-Supervised Transformers as Iterative Solution Improvers for Constraint Satisfactioncited by 6
  6. 2026Step-resolved data attribution for looped transformerscited by 5
  7. 2023Learning Reliable Logical Rules with SATNetcited by 3
  8. 2025Hyper-SET: Designing Transformers via Hyperspherical Energy Minimizationcited by 2
  9. 2025Throttling Web Agents Using Reasoning Gatescited by 2
  10. 2025Teaching Transformers to Solve Combinatorial Problems through Efficient Trial & Errorcited by 2
  11. 2025Generalizable Reasoning through Compositional Energy Minimizationcited by 2
  12. 2025Guided Discrete Diffusion for Constraint Satisfaction Problemscited by 1
9 more
  1. 2026Beyond Test-Time Memory: State-Space Optimal Control for LLM Reasoningcited by 1
  2. 2026Large Neighborhood Search meets Iterative Neural Constraint Heuristicscited by 1
  3. 2025Scaling Neuro-symbolic Problem Solving: Solver-Free Learning of Constraints and Objectivescited by 0
  4. 2026Imitation Learning for Combinatorial Optimisation under Uncertaintycited by 0
  5. 2026Winfree Oscillatory Neural Networkcited by 0
  6. 2026Blocked Gibbs meets Diffusion Transformers: Unsupervised Learning for Constraint Optimizationcited by 0
  7. 2026Position: Certified Correctness in Neural Constraint Reasoning Requires Symbolic Integrationcited by 0
  8. 2026Dissecting Hierarchical Reasoning Models: A Mechanistic Studycited by 0
  9. 2026The Geometry of Logic: Stratification Induces Semantic Structure and Robust Reasoningcited by 0