# Towards Trustworthy AI Software Development Assistance

arXiv: https://arxiv.org/abs/2312.09126 · HTML: https://mlsanity.com/paper/2312.09126/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 44 references that are in the map.
- First posted on arXiv 14 Dec 2023.

## Builds on (most cited first, 17 of 17)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Graph Neural Networks: A Review of Methods and Applications](https://mlsanity.com/paper/1812.08434/) (2018)
- [Competition-Level Code Generation with AlphaCode](https://mlsanity.com/paper/2203.07814/) (2022)
- [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://mlsanity.com/paper/2002.08155/) (2020)
- [CodeSearchNet Challenge: Evaluating the State of Semantic Code Search](https://mlsanity.com/paper/1909.09436/) (2019)
- [GraphCodeBERT: Pre-training Code Representations with Data Flow](https://mlsanity.com/paper/2009.08366/) (2020)
- [CodeBLEU: a Method for Automatic Evaluation of Code Synthesis](https://mlsanity.com/paper/2009.10297/) (2020)
- [Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions](https://mlsanity.com/paper/2108.09293/) (2021)
- [A Practical Guide to Multi-Objective Reinforcement Learning and Planning](https://mlsanity.com/paper/2103.09568/) (2021)
- [Do Users Write More Insecure Code with AI Assistants?](https://mlsanity.com/paper/2211.03622/) (2022)
- [ReACC: A Retrieval-Augmented Code Completion Framework](https://mlsanity.com/paper/2203.07722/) (2022)
- [Code Prediction by Feeding Trees to Transformers](https://mlsanity.com/paper/2003.13848/) (2020)
- [LLMSecEval: A Dataset of Natural Language Prompts for Security Evaluations](https://mlsanity.com/paper/2303.09384/) (2023)
- [What Do They Capture? -- A Structural Analysis of Pre-Trained Language Models for Source Code](https://mlsanity.com/paper/2202.06840/) (2022)
- [A Toolkit for Generating Code Knowledge Graphs](https://mlsanity.com/paper/2002.09440/) (2020)
- [CryptoGuard: High Precision Detection of Cryptographic Vulnerabilities in Massive-sized Java Projects](https://mlsanity.com/paper/1806.06881/) (2018)

## Built on by (most cited first, 4 of 4)

- [Towards More Trustworthy and Interpretable LLMs for Code through Syntax-Grounded Explanations](https://mlsanity.com/paper/2407.08983/) (2024)
- [Problem Solving Through Human-AI Preference-Based Cooperation](https://mlsanity.com/paper/2408.07461/) (2024)
- [textttReMind: Understanding Deductive Code Reasoning in LLMs](https://mlsanity.com/paper/2511.00488/) (2025)
- [Mapping the Trust Terrain: LLMs in Software Engineering -- Insights and Perspectives](https://mlsanity.com/paper/2503.13793/) (2025)

## Abstract (arXiv)

It is expected that in the near future, AI software development assistants will play an important role in the software industry. However, current software development assistants tend to be unreliable, often producing incorrect, unsafe, or low-quality code. We seek to resolve these issues by introducing a holistic architecture for constructing, training, and using trustworthy AI software development assistants. In the center of the architecture, there is a foundational LLM trained on datasets representative of real-world coding scenarios and complex software architectures, and fine-tuned on code quality criteria beyond correctness. The LLM will make use of graph-based code representations for advanced semantic comprehension. We envision a knowledge graph integrated into the system to provide up-to-date background knowledge and to enable the assistant to provide appropriate explanations. Finally, a modular framework for constrained decoding will ensure that certain guarantees (e.g., for correctness and security) hold for the generated code.