OpenCodeReview: Determinism over Non-Determinism for Cost-Effective Agent-Based Code Review
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
LLM-based code review agents promise scalable, always-on review, yet current systems suffer from two intertwined weaknesses: (1) non-determinism--unbounded tool use makes review outcomes unstable, and (2) context locality--the reviewer's access remains bounded to the diff, capping discoverable issue depth. Both give rise to three challenges: misaligned context retrieval, a coherence-efficiency trade-off in multi-file pull requests, and hallucinated comments that erode trust. To address these, we introduce OpenCodeReview, built on deterministic engineering for uncertain agents: rather than granting maximal freedom, we inject determinism at three deliberate pipeline points. Rule-Guided Dispatch uses a multi-layer rule system to deterministically select files and review criteria, eliminating variability of agent-driven triage. Grounded File Review replaces free-form exploration with a curated tool set exposed through a ReAct loop, while file-level parallel SubAgents balance context coherence against efficiency and recover cross-file dependencies on demand. Independent Reflection introduces a falsification-first filter under an asymmetric information boundary--the reflector sees only the diff, not the agent's tool-augmented exploration--removing hallucinated comments without self-reinforcing bias, improving precision while preserving recall. On AACR-Bench (200 real-world PRs, 10 languages, 1,505 expert-verified comments), OpenCodeReview outperforms mainstream coding agents (e.g., Claude Code and Codex) across six LLM backends, achieving up to 2.17x higher SEM-F1 (25.10% vs. 11.57%) while consuming 5-15x fewer tokens. We open-source OpenCodeReview at https://github.com/alibaba/open-code-review.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 15 of its 32 references that are in the map. First posted on arXiv 10 Aug 2026.
Builds on
15 of 32 references.
Closest ancestors · 3
Same direction, newest first
- 2026From Industry Claims to Empirical Reality: An Empirical Study of Code Review Agents in Pull Requests3 Apr
- 2025RepoAudit: An Autonomous LLM-Agent for Repository-Level Code Auditing30 Jan
- 2023ChatDev: Communicative Agents for Software Development16 Jul
Other references · 12
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
- 2022Automating Code Review Activities by Large-Scale Pre-trainingcited by 98
- 2025Automated Hypothesis Validation with Agentic Sequential Falsificationscited by 40
- 2025Efficient Agents: Building Effective Agents While Reducing Costcited by 32
- 2025LAURA: Enhancing Code Review Generation with Context-Enriched Retrieval-Augmented LLMcited by 8
- 2024Prompting and Fine-tuning Large Language Models for Automated Code Review Comment Generationcited by 7
- 2025Retrieval-Augmented Code Review Comment Generationcited by 6
- 2026AACR-Bench: Evaluating Automatic Code Review with Holistic Repository-Level Contextcited by 5
- 2026How Consistent Are LLM Agents? Measuring Behavioral Reproducibility in Multi-Step Tool-Calling Pipelinescited by 5
- 2025Fine-Tuning LLMs to Analyze Multiple Dimensions of Code Review: A Maximum Entropy Regulated Long Chain-of-Thought Approachcited by 1
- 2026Improving LLM-Based Go Code Review through Issue-List Generation and Context Augmentationcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026CR-Bench: Evaluating the Real-World Utility of AI Code Review Agents
- 2026Understanding Dominant Themes in Reviewing Agentic AI-authored Code
- 2026SWE-Review: Closing the Loop on Issue Resolution with Agentic Code Review
- 2026Adversarial Review: Structured Disagreement for Grounded Agentic Code Review
- 2026From Code Review to Code Critique: Intent, Drift, and Spotlight for AI-Generated Diffs at Scale
- 2025LoCoBench-Agent: An Interactive Benchmark for LLM Agents in Long-Context Software Engineering
- 2025SWE-PolyBench: A multi-language benchmark for repository level evaluation of coding agents
- 2026Self-Improving AI Coding Agents Through Accumulated Behavioral Rules: A Closed-Loop Framework