# Field-Aware Agent Skill Retrieval

arXiv: https://arxiv.org/abs/2608.02880 · HTML: https://mlsanity.com/paper/2608.02880/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 11 references that are in the map.
- First posted on arXiv 3 Aug 2026.

## Builds on (most cited first, 7 of 7)

- [Voyager: An Open-Ended Embodied Agent with Large Language Models](https://mlsanity.com/paper/2305.16291/) (2023)
- [Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models](https://mlsanity.com/paper/2506.05176/) (2025)
- [Retrieval Models Aren't Tool-Savvy: Benchmarking Tool Retrieval for Large Language Models](https://mlsanity.com/paper/2503.01763/) (2025)
- [Skill Retrieval Augmentation for Agentic AI](https://mlsanity.com/paper/2604.24594/) (2026)
- [SkillRet: A Large-Scale Benchmark for Skill Retrieval in LLM Agents](https://mlsanity.com/paper/2605.05726/) (2026)
- [More Skills, Worse Agents? Skill Shadowing Degrades Performance When Expanding Skill Libraries](https://mlsanity.com/paper/2605.24050/) (2026)
- [Multi-Field Adaptive Retrieval](https://mlsanity.com/paper/2410.20056/) (2024)

## Built on by (most cited first, 4 of 4)

- [What Does a Skill Actually Do? Estimands and Evaluation Validity for Tool and Skill Use in LLM Agents: A Critical Review](https://mlsanity.com/paper/2609.33153/) (2026)
- [AnyJev Technical Report](https://mlsanity.com/paper/2610.00831/) (2026)
- [SkillFM: Generating Skills for LLM Agents via Latent Flow Matching](https://mlsanity.com/paper/2609.39382/) (2026)
- [What Can Component-Replacement Evidence Establish? A Critical Scoping Review of Local Decisions in LLM Agents](https://mlsanity.com/paper/2609.39989/) (2026)

## Abstract (arXiv)

As lifelong learning agents accumulate lifelong growing skill banks, retrieving the correct skill becomes an increasingly important bottleneck. Most current skill retrieval methods treat each skill as one flat document by concatenating fields such as the name, description, and body. However, skills are naturally structured, multi-field objects, where each field provides different information about when and how the skill should be used. In this work, we study whether preserving this structure improves skill retrieval. We represent each skill as its separate components, and compute sparse and dense similarities for each field independently, exposing a naturally tensorized, field-aware representation of the skill bank. We then combine these field-level scores either with uniform weights or with a small learned MLP. Across two different skill retrieval benchmarks, SkillRet and SRA-Bench, we find that keeping fields separate improves hybrid retrieval, and learning over the field-level scores gives the strongest and most consistent results. Our field-aware MLP reaches 77.95 Recall@10 on SkillRet and 83.78 Recall@10 on SRA-Bench, outperforming the corresponding concatenated learned baselines. We also find that the advantage grows as the skill bank becomes larger, suggesting that field-aware skill retrieval becomes especially useful in the setting where retrieval is most difficult. Our results show that skill representation itself matters, and that simply preserving the structure already present in skill files can substantially improve retrieval.