mlsanityData to 14 Oct 2026

SWE-Sharp-Bench: A Reproducible Benchmark for C# Software Engineering Tasks

arXiv 2511.02352 ↗First posted 4 Nov 2025cs.SEcited by 4

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

AI coding agents have shown great progress on Python software engineering benchmarks like SWE-Bench, and for other languages like Java and C in benchmarks like Multi-SWE-Bench. However, C# -- a prominent enterprise language ranking #5 in the TIOBE index -- remains absent from such benchmarks. We introduce SWE-Sharp-Bench, a reproducible software engineering benchmark for C# featuring 150 instances from 17 repositories. Evaluating identical model-agent configurations across languages reveals a significant performance gap: while 70% of Python tasks in SWE-Bench Verified are solved, only 40% of our C# tasks are resolved. We open-source SWE-Sharp-Bench and our entire curation pipeline.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. First posted on arXiv 4 Nov 2025.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026SWE-Bench ProMax: Benchmarking Agents on Large-Scale Multilingual Code Refactoring
  2. 2025SWE-PolyBench: A multi-language benchmark for repository level evaluation of coding agents
  3. 2025SWE-EVO: Benchmarking Coding Agents in Long-Horizon Software Evolution Scenarios
  4. 2026SWE Atlas: Benchmarking Coding Agents Beyond Issue Resolution
  5. 2025SWE-Compass: Towards Unified Evaluation of Agentic Coding Abilities for Large Language Models
  6. 2025Saving SWE-Bench: A Benchmark Mutation Approach for Realistic Agent Evaluation
  7. 2026SWE-Bench Mobile: Can Large Language Model Agents Develop Industry-Level Mobile Applications?
  8. 2026SWE-Bench Pro Verified: A Reliable Benchmark for Software Engineering Agents