mlsanityData to 14 Oct 2026

CUCo: An Agentic Framework for Compute and Communication Co-design

arXiv 2603.02376 ↗First posted 2 Mar 2026cs.DC cs.AR cs.LGcited by 2

On the map: LLM agents and tool use → Software engineering coding agents

also in 1 more
Abstract (arXiv)

Computation and communication in distributed LLM training and inference are traditionally optimized in isolation; expert-crafted systems such as DeepEP, FLUX, and TokenWeave show the potential of co-design but require deep systems expertise and hardware-specific tuning; CUCo is an agentic framework that automates compute-communication co-design of CUDA kernels by combining a structured design-space formalization with a correctness-first fast-path agent for reliable baselines and an evolution-driven slow-path agent for high-performance strategies, achieving up to 1.57x speedup across four multi-GPU workloads and discovering a two-stream overlap strategy on a DeepSeek-V3 MoE layer that hides dispatch behind local compute at an LLM inference cost under 10 per workload.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 29 of its 46 references that are in the map. First posted on arXiv 2 Mar 2026.

Builds on

29 of 46 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025CudaForge: An Agent Framework with Hardware Feedback for CUDA Kernel Optimization23 Oct
  2. 2025STARK: Strategic Team of Agents for Refining Kernels19 Oct
  3. 2025Towards Robust Agentic CUDA Kernel Benchmarking, Verification, and Optimization16 Sep

Other references · 26

  1. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  2. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  3. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  4. 2024FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precisioncited by 519
  5. 2023Ring Attention with Blockwise Transformers for Near-Infinite Contextcited by 457
  6. 2015Illuminating search spaces by mapping elitescited by 425
  7. 2014cuDNN: Efficient Primitives for Deep Learningcited by 245
  8. 2025KernelBench: Can LLMs Write Efficient GPU Kernels?cited by 225
  9. 2020Ansor: Generating High-Performance Tensor Programs for Deep Learningcited by 201
  10. 2025ShinkaEvolve: Towards Open-Ended And Sample-Efficient Program Evolutioncited by 174
  11. 2024NanoFlow: Towards Optimal Large Language Model Serving Throughputcited by 90
  12. 2025Kevin: Multi-Turn RL for Generating CUDA Kernelscited by 78
14 more
  1. 2025CUDA-L1: Improving CUDA Optimization via Contrastive Reinforcement Learningcited by 61
  2. 2024FLUX: Fast Software-based Communication Overlap On GPUs Through Kernel Fusioncited by 57
  3. 2023ASTRA-sim2.0: Modeling Hierarchical Networks and Disaggregated Systems for Large-model Training at Scalecited by 47
  4. 2021Breaking the Computation and Communication Abstraction Barrier in Distributed Machine Learning Workloadscited by 40
  5. 2020Enabling Compute-Communication Overlap in Distributed Deep Learning Training Platformscited by 25
  6. 2025TileLink: Generating Efficient Compute-Communication Overlapping Kernels using Tile-Centric Primitivescited by 24
  7. 2020Automatic Horizontal Fusion for GPU Kernelscited by 17
  8. 2023Optimizing Distributed ML Communication with Fused Computation-Collective Operationscited by 15
  9. 2025TokenWeave: Efficient Compute-Communication Overlap for Distributed LLM Inferencecited by 14
  10. 2025EvoEngineer: Mastering Automated CUDA Kernel Code Evolution with Large Language Modelscited by 13
  11. 2025ConCuR: Conciseness Makes State-of-the-Art Kernel Generationcited by 13
  12. 2025ParallelKittens: Systematic and Practical Simplification of Multi-GPU AI Kernelscited by 12
  13. 2025Efficient and Adaptable Overlapping for Computation and Communication via Signaling and Reorderingcited by 10
  14. 2024Domino: Eliminating Communication in LLM Training via Generic Tensor Slicing and Overlappingcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Ave: Guiding Agentic GPU Optimization Using Data-Flow Invariantscited by 0
  2. 2026ATTUNE: Agentic Throughput Tuning for LLM Trainingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026CUDA Agent: Large-Scale Agentic RL for High-Performance CUDA Kernel Generation
  2. 2026KernelBlaster: Continual Cross-Task CUDA Optimization via Memory-Augmented In-Context Reinforcement Learning
  3. 2026Kernel Forge: An Agent Harness for LLM-based Generation and Optimization of CUDA Kernels
  4. 2025Astra: A Multi-Agent System for GPU Kernel Performance Optimization
  5. 2026Agentic Kernel Optimization: Generating State-of-the-Art GPU Kernels Without Hand-Written CUDA
  6. 2026CuTeGen: An LLM-Based Agentic Framework for Generation and Optimization of High-Performance GPU Kernels using CuTe
  7. 2025David vs. Goliath: Can Small Models Win Big with Agentic AI in Hardware Design?
  8. 2026KernelOPT: Dispatch-Aware Agentic Search for GPU Kernel Optimization