Enhancing Code Consistency in AI Research with Large Language Models and Retrieval-Augmented Generation
On the map: Retrieval-augmented generation → RAG for code
Abstract (arXiv)
Ensuring that code accurately reflects the algorithms and methods described in research papers is critical for maintaining credibility and fostering trust in AI research. This paper presents a novel system designed to verify code implementations against the algorithms and methodologies outlined in corresponding research papers. Our system employs Retrieval-Augmented Generation to extract relevant details from both the research papers and code bases, followed by a structured comparison using Large Language Models. This approach improves the accuracy and comprehensiveness of code implementation verification while contributing to the transparency, explainability, and reproducibility of AI research. By automating the verification process, our system reduces manual effort, enhances research credibility, and ultimately advances the state of the art in code verification.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in RAG for code, one of 15 directions in Retrieval-augmented generation. Builds on 6 of its 12 references that are in the map. First posted on arXiv 2 Feb 2025.
Builds on
6 of 12 references.
References · 6
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2020Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)cited by 318
- 2019A Step Toward Quantifying Independently Reproducible Machine Learning Researchcited by 77
- 2023LLaMA-Reviewer: Advancing Code Review Automation with Large Language Models through Parameter-Efficient Fine-Tuningcited by 47
- 2024Reproducibility in Machine Learning-based Research: Overview, Barriers and Driverscited by 34
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Are you Synthesizing or Recalling? Evaluating LLMs on Algorithmic Code Retrieval
- 2025SciReplicate-Bench: Benchmarking LLMs in Agent-driven Algorithmic Reproduction from Research Papers
- 2026Context-Augmented Code Generation: How Product Context Improves AI Coding Agent Decision Compliance by 49%
- 2026Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap
- 2025Enhancing LLM Code Generation: A Systematic Evaluation of Multi-Agent Collaboration and Runtime Debugging for Improved Accuracy, Reliability, and Latency
- 2026Between the Commits: Process, Error, and Claim Reliability in a Wholly AI-Authored Codebase
- 2025An Empirical Study of Retrieval-Augmented Code Generation: Challenges and Opportunities
- 2025CodeMirage: A Multi-Lingual Benchmark for Detecting AI-Generated and Paraphrased Source Code from Production-Level LLMs