# Cognitive-YOLO: LLM-Driven Architecture Synthesis from First Principles of Data for Object Detection

arXiv: https://arxiv.org/abs/2512.12281 · HTML: https://mlsanity.com/paper/2512.12281/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 32 references that are in the map.
- First posted on arXiv 13 Dec 2025.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [YOLO-World: Real-Time Open-Vocabulary Object Detection](https://mlsanity.com/paper/2401.17270/) (2024)
- [Agentic Retrieval-Augmented Generation: A Survey on Agentic RAG](https://mlsanity.com/paper/2501.09136/) (2025)
- [LLMatic: Neural Architecture Search via Large Language Models and Quality Diversity Optimization](https://mlsanity.com/paper/2306.01102/) (2023)
- [Zen-NAS: A Zero-Shot NAS for High-Performance Deep Image Recognition](https://mlsanity.com/paper/2102.01063/) (2021)
- [DEIM: DETR with Improved Matching for Fast Convergence](https://mlsanity.com/paper/2412.04234/) (2024)
- [LLM Performance Predictors are good initializers for Architecture Search](https://mlsanity.com/paper/2310.16712/) (2023)
- [LPZero: Language Model Zero-cost Proxy Search from Zero](https://mlsanity.com/paper/2410.04808/) (2024)
- [LLM-Guided Evolution: An Autonomous Model Optimization for Object Detection](https://mlsanity.com/paper/2504.02280/) (2025)
- [PhaseNAS: Language-Model Driven Architecture Search with Dynamic Phase Adaptation](https://mlsanity.com/paper/2507.20592/) (2025)
- [Beyond CNNs: Efficient Fine-Tuning of Multi-Modal LLMs for Object Detection on Low-Data Regimes](https://mlsanity.com/paper/2510.08589/) (2025)

## Built on by (most cited first, 1 of 1)

- [A Hybrid Vision-Language Architecture for Automated Defect Reasoning and Report Generation in Industrial Inspection](https://mlsanity.com/paper/2605.26533/) (2026)

## Abstract (arXiv)

General-purpose object detection algorithms face a domain-shift bottleneck when deployed in vertical scenarios, and manually customizing a network architecture relies heavily on expert experience with high trial-and-error cost. To address these problems, we present Cognitive-YOLO, an adaptive object-detection model synthesis system driven by a large language model (LLM) collaborating with an autonomous agent. Cognitive-YOLO builds a fully automated Analyze--Synthesize--Compile pipeline. First, a data-profiling module quantitatively extracts dataset features, guiding a ReAct-paradigm agent to autonomously retrieve matching network modules from a curated component library. Second, the LLM performs logical reasoning and topological assembly conditioned on the data features, producing a specification-compliant neural architecture description. Finally, a backend hybrid compiler dynamically instantiates the model, closing the loop with end-to-end training and evaluation scheduling. Cross-validation on datasets including rail-surface defects and rice diseases shows that the lightweight models synthesized by Cognitive-YOLO markedly compress the parameter count (down to 1.9M) while matching or surpassing mainstream baselines such as YOLOv11n in mAP@0.5:0.95, effectively balancing model compactness and feature-representation capability.