Which Features are Learned by CodeBert: An Empirical Study of the BERT-based Source Code Representation Learning
Not on the map. Why?
Abstract (arXiv)
The Bidirectional Encoder Representations from Transformers (BERT) were proposed in the natural language process (NLP) and shows promising results. Recently researchers applied the BERT to source-code representation learning and reported some good news on several downstream tasks. However, in this paper, we illustrated that current methods cannot effectively understand the logic of source codes. The representation of source code heavily relies on the programmer-defined variable and function names. We design and implement a set of experiments to demonstrate our conjecture and provide some insights for future works.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 6 of its 16 references that are in the map. First posted on arXiv 20 Jan 2023.
Builds on
6 of 16 references.
References · 6
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,310
- 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 609
- 2021PalmTree: Learning an Assembly Language Model for Instruction Embeddingcited by 49
- 2019Using Deep Learning to Solve Computer Security Challenges: A Surveycited by 2
- 2021Spotting Silent Buffer Overflows in Execution Trace through Graph Neural Network Assisted Data Flow Analysiscited by 2
Built on by
Citing papers per year: 2025: 2.
2 papers.