# An Empirical Study on Strong-Weak Model Collaboration for Repo-level Code Generation

arXiv: https://arxiv.org/abs/2505.20182 · HTML: https://mlsanity.com/paper/2505.20182/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 56 references that are in the map.
- First posted on arXiv 26 May 2025.

## Builds on (most cited first, 18 of 18)

- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance](https://mlsanity.com/paper/2305.05176/) (2023)
- [Universal Self-Consistency for Large Language Model Generation](https://mlsanity.com/paper/2311.17311/) (2023)
- [The Danger of Overthinking: Examining the Reasoning-Action Dilemma in Agentic Tasks](https://mlsanity.com/paper/2502.08235/) (2025)
- [Large Language Model Cascades with Mixture of Thoughts Representations for Cost-efficient Reasoning](https://mlsanity.com/paper/2310.03094/) (2023)
- [CodeRAG-Bench: Can Retrieval Augment Code Generation?](https://mlsanity.com/paper/2406.14497/) (2024)
- [RepoGraph: Enhancing AI Software Engineering with Repository-level Code Graph](https://mlsanity.com/paper/2410.14684/) (2024)
- [Small Language Models Need Strong Verifiers to Self-Correct Reasoning](https://mlsanity.com/paper/2404.17140/) (2024)
- [Harnessing Multiple Large Language Models: A Survey on LLM Ensemble](https://mlsanity.com/paper/2502.18036/) (2025)
- [Planning In Natural Language Improves LLM Search For Code Generation](https://mlsanity.com/paper/2409.03733/) (2024)
- [EcoAssistant: Using LLM Assistant More Affordably and Accurately](https://mlsanity.com/paper/2310.03046/) (2023)
- [GraphCodeAgent: Dual Graph-Guided LLM Agent for Retrieval-Augmented Repo-Level Code Generation](https://mlsanity.com/paper/2504.10046/) (2025)
- [Self-Resource Allocation in Multi-Agent LLM Systems](https://mlsanity.com/paper/2504.02051/) (2025)
- [Improving Large Models with Small models: Lower Costs and Better Performance](https://mlsanity.com/paper/2406.15471/) (2024)
- [CodeSteer: Symbolic-Augmented Language Models via Code/Text Guidance](https://mlsanity.com/paper/2502.04350/) (2025)
- [Can We Afford The Perfect Prompt? Balancing Cost and Accuracy with the Economical Prompting Index](https://mlsanity.com/paper/2412.01690/) (2024)

## Built on by (most cited first, 1 of 1)

- [CentaurBench: Benchmarking LLM Capabilities on Augmenting vs. Automating Real-World Work Tasks](https://mlsanity.com/paper/2608.18554/) (2026)

## Abstract (arXiv)

We study cost-efficient collaboration between strong and weak language models for repository-level code generation, where the weak model handles simpler tasks at lower cost, and the most challenging tasks are delegated to the strong model. While many works propose architectures for this task, few analyze performance relative to cost. We evaluate a broad spectrum of collaboration strategies: context-based, pipeline-based, and dynamic, on GitHub issue resolution. Our most effective collaborative strategy achieves equivalent performance to the strong model while reducing the cost by 40%. Based on our findings, we offer actionable guidelines for choosing collaboration strategies under varying budget and performance constraints. Our results show that strong-weak collaboration substantially boosts the weak model's performance at a fraction of the cost, pipeline and context-based methods being most efficient. We release the code for our work at https://github.com/shubhamrgandhi/codegen-strong-weak-collab.