mlsanityData to 7 Oct 2026

A Survey of Deep Learning Models for Structural Code Understanding

arXiv 2205.01293 ↗First posted 3 May 2022cs.SE cs.AI cs.GLcited by 4

Not on the map. Why?

Abstract (arXiv)

In recent years, the rise of deep learning and automation requirements in the software industry has elevated Intelligent Software Engineering to new heights. The number of approaches and applications in code understanding is growing, with deep learning techniques being used in many of them to better capture the information in code data. In this survey, we present a comprehensive overview of the structures formed from code data. We categorize the models for understanding code in recent years into two groups: sequence-based and graph-based models, further make a summary and comparison of them. We also introduce metrics, datasets and the downstream tasks. Finally, we make some suggestions for future research in structural code understanding field.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 105 of its 182 references that are in the map. First posted on arXiv 3 May 2022.

Builds on

105 of 182 references.

References · 105

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  4. 2017Graph Attention Networkscited by 8,854
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  6. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  7. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  8. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  9. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  10. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  11. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,906
  12. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
38 more
  1. 2018Self-Attention with Relative Position Representationscited by 1,112
  2. 2015Teaching Machines to Read and Comprehendcited by 1,039
  3. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
  4. 2015Gated Graph Sequence Neural Networkscited by 792
  5. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 732
  6. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 638
  7. 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 600
  8. 2020CodeBLEU: a Method for Automatic Evaluation of Code Synthesiscited by 468
  9. 2015A Neural Attention Model for Abstractive Sentence Summarizationcited by 394
  10. 2015Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networkscited by 305
  11. 2019Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networkscited by 266
  12. 2018code2vec: Learning Distributed Representations of Codecited by 249
  13. 2020IntelliCode Compose: Code Generation Using Transformercited by 225
  14. 2017A Survey of Machine Learning for Big Code and Naturalnesscited by 220
  15. 2017Learning to Represent Programs with Graphscited by 219
  16. 2021CodeNet: A Large-Scale AI for Code Dataset for Learning a Diversity of Coding Taskscited by 199
  17. 2018code2seq: Generating Sequences from Structured Representations of Codecited by 177
  18. 2018Learning to Mine Aligned Code and Natural Language Pairs from Stack Overflowcited by 165
  19. 2014A Convolutional Neural Network for Modelling Sentencescited by 163
  20. 2020Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processingcited by 161
  21. 2018SequenceR: Sequence-to-Sequence Learning for End-to-End Program Repaircited by 144
  22. 2020A Transformer-based Approach for Source Code Summarizationcited by 129
  23. 2018The Adverse Effects of Code Duplication in Machine Learning Models of Codecited by 123
  24. 2018Mapping Language to Code in Programmatic Contextcited by 104
  25. 2018Improving Automatic Source Code Summarization via Deep Reinforcement Learningcited by 99
  26. 2018NL2Bash: A Corpus and Semantic Parser for Natural Language Interface to the Linux Operating Systemcited by 97
  27. 2020Graph-based, Self-Supervised Program Repair from Diagnostic Feedbackcited by 93
  28. 2017Abstract Syntax Networks for Code Generation and Semantic Parsingcited by 92
  29. 2016A Convolutional Attention Network for Extreme Summarization of Source Codecited by 90
  30. 2020Retrieval-Augmented Generation for Code Summarization via Hybrid GNNcited by 89
  31. 2018TRANX: A Transition-based Neural Abstract Syntax Parser for Semantic Parsing and Code Generationcited by 88
  32. 2020Improved Code Summarization via a Graph Neural Networkcited by 79
  33. 2019A Neural Model for Generating Natural Language Summaries of Program Subroutinescited by 76
  34. 2021CoSQA: 20,000+ Web Queries for Code Search and Question Answeringcited by 74
  35. 2020Code Prediction by Feeding Trees to Transformerscited by 70
  36. 2017Neural Network-based Graph Embedding for Cross-Platform Binary Code Similarity Detectioncited by 65
  37. 2021Language-Agnostic Representation Learning of Source Code from Structure and Contextcited by 63
  38. 2018Generative Code Modeling with Graphscited by 59

50 most cited of 105 All 105 (JSON)

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 1; 2024: 2.

4 papers.

  1. 2022A Survey on Pretrained Language Models for Neural Code Intelligencecited by 13
  2. 2024Evaluation of large language models for assessing code maintainabilitycited by 5
  3. 2023Neural Machine Translation for Code Generationcited by 3
  4. 2024SimBench: A Framework for Evaluating and Diagnosing LLM-Based Digital-Twin Generation for Multi-Physics Simulationcited by 3