mlsanityData to 7 Oct 2026

Exploiting Code Symmetries for Learning Program Semantics

arXiv 2308.03312 ↗First posted 7 Aug 2023cs.LG cs.CR cs.PLcited by 9

Not on the map. Why?

Abstract (arXiv)

This paper tackles the challenge of teaching code semantics to Large Language Models (LLMs) for program analysis by incorporating code symmetries into the model architecture. We introduce a group-theoretic framework that defines code symmetries as semantics-preserving transformations, where forming a code symmetry group enables precise and efficient reasoning of code semantics. Our solution, SymC, develops a novel variant of self-attention that is provably equivariant to code symmetries from the permutation group defined over the program dependence graph. SymC obtains superior performance on five program analysis tasks, outperforming state-of-the-art code models without any pre-training. Our results suggest that code LLMs that encode the code structural prior via the code symmetry group generalize better and faster.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 50 of its 91 references that are in the map. First posted on arXiv 7 Aug 2023.

Builds on

50 of 91 references.

References · 50

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  4. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
  5. 2016Group Equivariant Convolutional Networkscited by 1,154
  6. 2016Geometric deep learning: going beyond Euclidean datacited by 1,042
  7. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
  8. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
  9. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 728
  10. 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 600
  11. 2019Deep Ensembles: A Loss Landscape Perspectivecited by 438
  12. 2021Unified Pre-training for Program Understanding and Generationcited by 361
38 more
  1. 2022UniXcoder: Unified Cross-Modal Pre-training for Code Representationcited by 355
  2. 2018Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understandingcited by 304
  3. 2018Towards a Definition of Disentangled Representationscited by 294
  4. 2018code2vec: Learning Distributed Representations of Codecited by 249
  5. 2017Learning SO(3) Equivariant Representations with Spherical CNNscited by 244
  6. 2017Learning to Represent Programs with Graphscited by 219
  7. 2017Neural-Symbolic Learning and Reasoning: A Survey and Interpretationcited by 195
  8. 2022Graph neural networks for materials science and chemistrycited by 181
  9. 2018code2seq: Generating Sequences from Structured Representations of Codecited by 177
  10. 2016Harnessing Deep Neural Networks with Logic Rulescited by 135
  11. 2020Incorporating Symmetry into Deep Dynamics Models for Improved Generalizationcited by 123
  12. 2020Lorentz Group Equivariant Neural Network for Particle Physicscited by 120
  13. 2020LieTransformer: Equivariant self-attention for Lie Groupscited by 100
  14. 2022ReCode: Robustness Evaluation of Code Generation Modelscited by 100
  15. 2017Structured Attention Networkscited by 95
  16. 2016A Convolutional Attention Network for Extreme Summarization of Source Codecited by 90
  17. 2021Automatic Symmetry Discovery with Lie Algebra Convolutional Networkcited by 82
  18. 2020Code Prediction by Feeding Trees to Transformerscited by 70
  19. 2021DOBF: A Deobfuscation Pre-Training Objective for Programming Languagescited by 68
  20. 2017Neural Network-based Graph Embedding for Cross-Platform Binary Code Similarity Detectioncited by 65
  21. 2020Group Equivariant Stand-Alone Self-Attention For Visioncited by 63
  22. 2023Scallop: A Language for Neurosymbolic Programmingcited by 63
  23. 2018DeepSphere: Efficient spherical Convolutional Neural Network with HEALPix sampling for cosmological applicationscited by 61
  24. 2019Adversarial Examples for Models of Codecited by 61
  25. 2019TreeGen: A Tree-Based Transformer Architecture for Code Generationcited by 58
  26. 2018Structured Neural Summarizationcited by 53
  27. 2021Neural Program Repair with Execution-based Backpropagationcited by 50
  28. 2021PalmTree: Learning an Assembly Language Model for Instruction Embeddingcited by 49
  29. 2020Semantic Robustness of Models of Source Codecited by 43
  30. 2020On the Generalizability of Neural Program Models with respect to Semantic-Preserving Program Transformationscited by 41
  31. 2020Learning to Execute Programs with Instruction Pointer Attention Graph Neural Networkscited by 35
  32. 2020Trex: Learning Execution Semantics from Micro-Traces for Binary Similaritycited by 34
  33. 2023TRACED: Execution-aware Pre-training for Source Codecited by 27
  34. 2022Equi-Tuning: Group Equivariant Fine-Tuning of Pretrained Modelscited by 22
  35. 2022Static Prediction of Runtime Errors by Learning to Execute Programs with External Resource Descriptionscited by 11
  36. 2022Two Sides of the Same Coin: Exploiting the Impact of Identifiers in Neural Code Comprehensioncited by 9
  37. 2023LExecutor: Learning-Guided Executioncited by 7
  38. 2022Black-box Attacks Against Neural Binary Function Detectioncited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 5; 2026: 2.

9 papers.

  1. 2024Large Language Models for Cyber Security: A Systematic Literature Reviewcited by 65
  2. 2024LLM4Vuln: A Unified Evaluation Framework for Decoupling and Enhancing LLMs' Vulnerability Reasoningcited by 57
  3. 2025TIF: Learning Temporal Invariance in Android Malware Detectorscited by 4
  4. 2025ObscuraCoder: Powering Efficient Code LM Pre-Training Via Obfuscation Groundingcited by 4
  5. 2025Rethinking LLM-Based RTL Code Optimization Via Timing Logic Metamorphosiscited by 1
  6. 2025Large Language Models in Cybersecurity: Applications, Vulnerabilities, and Defense Techniquescited by 0
  7. 2025Retrofit: Continual Learning with Controlled Forgetting for Binary Security Detection and Analysiscited by 0
  8. 2026SemRep: Generative Code Representation Learning with Code Transformationscited by 0
  9. 2026What are the Right Symmetries for Formal Theorem Proving?cited by 0