# A Study of Variable-Role-based Feature Enrichment in Neural Models of Code

arXiv: https://arxiv.org/abs/2303.04942 · HTML: https://mlsanity.com/paper/2303.04942/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 35 references that are in the map.
- First posted on arXiv 8 Mar 2023.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://mlsanity.com/paper/2002.08155/) (2020)
- [CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation](https://mlsanity.com/paper/2102.04664/) (2021)
- [GraphCodeBERT: Pre-training Code Representations with Data Flow](https://mlsanity.com/paper/2009.08366/) (2020)
- [Unsupervised Translation of Programming Languages](https://mlsanity.com/paper/2006.03511/) (2020)
- [code2vec: Learning Distributed Representations of Code](https://mlsanity.com/paper/1803.09473/) (2018)
- [code2seq: Generating Sequences from Structured Representations of Code](https://mlsanity.com/paper/1808.01400/) (2018)
- [On the Generalizability of Neural Program Models with respect to Semantic-Preserving Program Transformations](https://mlsanity.com/paper/2008.01566/) (2020)
- [Multilingual training for Software Engineering](https://mlsanity.com/paper/2112.02043/) (2021)
- [Aroma: Code Recommendation via Structural Code Search](https://mlsanity.com/paper/1812.01158/) (2018)
- [InferCode: Self-Supervised Learning of Code Representations by Predicting Subtrees](https://mlsanity.com/paper/2012.07023/) (2020)
- [Understanding Neural Code Intelligence Through Program Simplification](https://mlsanity.com/paper/2106.03353/) (2021)
- [Learning Semantic Program Embeddings with Graph Interval Neural Network](https://mlsanity.com/paper/2005.09997/) (2020)
- [Structured Generative Models of Natural Source Code](https://mlsanity.com/paper/1401.0514/) (2014)
- [Towards Learning (Dis)-Similarity of Source Code from Program Contrasts](https://mlsanity.com/paper/2110.03868/) (2021)
- [Embedding Java Classes with code2vec: Improvements from Variable Obfuscation](https://mlsanity.com/paper/2004.02942/) (2020)
- [VarCLR: Variable Semantic Representation Pre-training via Contrastive Learning](https://mlsanity.com/paper/2112.02650/) (2021)
- [sk_p: a neural program corrector for MOOCs](https://mlsanity.com/paper/1607.02902/) (2016)
- [Two Sides of the Same Coin: Exploiting the Impact of Identifiers in Neural Code Comprehension](https://mlsanity.com/paper/2207.11104/) (2022)
- [Towards Demystifying Dimensions of Source Code Embeddings](https://mlsanity.com/paper/2008.13064/) (2020)
- [Code2Snapshot: Using Code Snapshots for Learning Representations of Source Code](https://mlsanity.com/paper/2111.01097/) (2021)

## Abstract (arXiv)

Although deep neural models substantially reduce the overhead of feature engineering, the features readily available in the inputs might significantly impact training cost and the performance of the models. In this paper, we explore the impact of an unsuperivsed feature enrichment approach based on variable roles on the performance of neural models of code. The notion of variable roles (as introduced in the works of Sajaniemi et al. [Refs. 1,2]) has been found to help students' abilities in programming. In this paper, we investigate if this notion would improve the performance of neural models of code. To the best of our knowledge, this is the first work to investigate how Sajaniemi et al.'s concept of variable roles can affect neural models of code. In particular, we enrich a source code dataset by adding the role of individual variables in the dataset programs, and thereby conduct a study on the impact of variable role enrichment in training the Code2Seq model. In addition, we shed light on some challenges and opportunities in feature enrichment for neural code intelligence models.