mlsanityData to 14 Oct 2026

CommitDistill: A Lightweight Knowledge-Centric Memory Layer for Software Repositories

arXiv 2605.18284 ↗First posted 18 May 2026cs.SE cs.AIcited by 0

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

Software repositories accumulate large amounts of unstructured knowledge in commit messages, pull-request discussions, and issue threads, but developers and AI coding assistants rarely reuse this history effectively. Recent work on typed-memory architectures for LLM agents (MemGPT, generative agents, and the PlugMem module of Yang et al.) argues that agent memory should be distilled, typed knowledge rather than raw interaction text. We adapt that stance to a software repository's own git history under a constrained regime: deterministic, dependency-free, local-only, no embeddings. We present CommitDistill, an open-source Python prototype that mines a local git history into typed knowledge units (Facts, Skills, Patterns) using deterministic regex and surfaces them through a TF-IDF retriever with a calibrated silence threshold (theta = 2.5) that abstains on out-of-distribution queries. The artefact is a trust-instrumented memory substrate: deterministic, no external service, inspectable plain-JSON store, tunable abstention. A case study on five public repositories spanning Python, JavaScript, C, and Java (25,000 commits, 1,167 extracted units) reports useful-precision 0.525 at Cohen's kappa = 0.633 on 40 dual-annotated Python units. The decisive finding is budget-constrained retrieval: at a 256-character per-query budget, CommitDistill reaches 0.750 hit-rate on a 12-query benchmark against BM25's 0.333 and git log --grep's 0.083. On a four-arm paired LLM-as-judge evaluation (n=200 time-travel bug-fixes, two judges) covering control, CommitDistill, a body-budget-matched CD-Hybrid, and BM25, no condition produces a statistically detectable lift over control on the headline mean and CD-Hybrid is indistinguishable from BM25 head-to-head. Extraction over 10,000 commits completes in under 4 seconds on a laptop. Source, annotations, baselines, and a reproducibility script accompany this paper.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 9 of its 23 references that are in the map. First posted on arXiv 18 May 2026.

Builds on

9 of 23 references.

References · 9

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  4. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  5. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
  6. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 634
  7. 2023Cognitive Architectures for Language Agentscited by 429
  8. 2022Productivity Assessment of Neural Code Completioncited by 98
  9. 2026PlugMem: A Task-Agnostic Plugin Memory Module for LLM Agentscited by 32

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Learning to Commit: Generating Organic Pull Requests via Online Repository Memory
  2. 2026Lore: Repurposing Git Commit Messages as a Structured Knowledge Protocol for AI Coding Agents
  3. 2026Why Git Is the Memory Solution for the Agentic Development Lifecycle
  4. 2026Agentic Repository Mining: A Multi-Task Evaluation
  5. 2026Agent Zero Memory: Provenance-Aware Long-Term Memory for LLM Agents
  6. 2025Prometheus: Towards Long-Horizon Codebase Navigation for Repository-Level Problem Solving
  7. 2026Ontology-Grounded Project Memory for Coding Agents
  8. 2026VibeMemBench: Evaluating Memory Systems for Coding Agents on Real Repository Coding Tasks