mlsanityData to 7 Oct 2026

Fix Bugs with Transformer through a Neural-Symbolic Edit Grammar

arXiv 2204.06643 ↗First posted 13 Apr 2022cs.LG cs.AI cs.PLcited by 8

Not on the map. Why?

Abstract (arXiv)

We introduce NSEdit (neural-symbolic edit), a novel Transformer-based code repair method. Given only the source code that contains bugs, NSEdit predicts an editing sequence that can fix the bugs. The edit grammar is formulated as a regular language, and the Transformer uses it as a neural-symbolic scripting interface to generate editing programs. We modify the Transformer and add a pointer network to select the edit locations. An ensemble of rerankers are trained to re-rank the editing sequences generated by beam search. We fine-tune the rerankers on the validation set to reduce over-fitting. NSEdit is evaluated on various code repair datasets and achieved a new state-of-the-art accuracy (24.04%) on the Tufano small dataset of the CodeXGLUE benchmark. NSEdit performs robustly when programs vary from packages to packages and when buggy programs are concrete. We conduct detailed analysis on our methods and demonstrate the effectiveness of each component.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 36 of its 57 references that are in the map. First posted on arXiv 13 Apr 2022.

Builds on

36 of 57 references.

References · 36

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  7. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  8. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  9. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
  10. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
  11. 2015Pointer Networkscited by 882
  12. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 732
24 more
  1. 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 600
  2. 2019Are Transformers universal approximators of sequence-to-sequence functions?cited by 404
  3. 2018Tune: A Research Platform for Distributed Model Selection and Trainingcited by 395
  4. 2019The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervisioncited by 369
  5. 2021Unified Pre-training for Program Understanding and Generationcited by 361
  6. 2016Professor Forcing: A New Algorithm for Training Recurrent Networkscited by 214
  7. 2019Stochastic Beams and Where to Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacementcited by 205
  8. 2019Facebook FAIR's WMT19 News Translation Task Submissioncited by 179
  9. 2018SequenceR: Sequence-to-Sequence Learning for End-to-End Program Repaircited by 144
  10. 2018An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translationcited by 106
  11. 2018A Retrieve-and-Edit Framework for Predicting Structured Outputscited by 73
  12. 2021Break-It-Fix-It: Unsupervised Learning for Program Repaircited by 71
  13. 2021CoTexT: Multi-task Learning with Code-Text Transformercited by 61
  14. 2021Self-Supervised Bug Detection and Repaircited by 59
  15. 2021A Syntax-Guided Edit Decoder for Neural Program Repaircited by 57
  16. 2020On the Generalizability of Neural Program Models with respect to Semantic-Preserving Program Transformationscited by 41
  17. 2019Getafix: Learning to Fix Bugs Automaticallycited by 40
  18. 2019Neural Program Repair by Jointly Learning to Localize and Repaircited by 39
  19. 2018Learning to Represent Editscited by 34
  20. 2019Learning to Fix Build Errors with Graph2Diff Neural Networkscited by 26
  21. 2021DeepDebug: Fixing Python Bugs Using Stack Traces, Backtranslation, and Code Skeletonscited by 17
  22. 2021Learning Structural Edits via Incremental Tree Transformationscited by 16
  23. 2017Siamese Neural Networks for One-shot detection of Railway Track Switchescited by 3
  24. 2018Train on Validation: Squeezing the Data Lemoncited by 1

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 2; 2025: 4.

8 papers.

  1. 2023RAP-Gen: Retrieval-Augmented Patch Generation with CodeT5 for Automatic Program Repaircited by 43
  2. 2025QualityFlow: An Agentic Workflow for Program Synthesis Controlled by LLM Quality Checkscited by 22
  3. 2025From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligencecited by 13
  4. 2024Neuro-Symbolic AI: Explainability, Challenges, and Future Trendscited by 10
  5. 2023Neural Machine Translation for Code Generationcited by 3
  6. 2025AuPair: Golden Example Pairs for Code Repaircited by 3
  7. 2024Bridging the Gap: Representation Spaces in Neuro-Symbolic AIcited by 0
  8. 2025Toward Neurosymbolic Program Comprehensioncited by 0