# MetaSynth: Multi-Agent Metadata Generation from Implicit Feedback in Black-Box Systems

arXiv: https://arxiv.org/abs/2510.01523 · HTML: https://mlsanity.com/paper/2510.01523/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Agentic RAG, agents and memory, one of 15 directions in Retrieval-augmented generation.
- Builds on 33 of its 42 references that are in the map.
- First posted on arXiv 1 Oct 2025.

## Placed in

- Retrieval-augmented generation / [Agentic RAG, agents and memory](https://mlsanity.com/rag/agentic-rag-agents-and-memory/)
- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 30 of 33)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Bias and Debias in Recommender System: A Survey and Future Directions](https://mlsanity.com/paper/2010.03240/) (2020)
- [Bridging Language and Items for Retrieval and Recommendation: Benchmarking LLMs as Semantic Encoders](https://mlsanity.com/paper/2403.03952/) (2024)
- [Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation](https://mlsanity.com/paper/1703.09902/) (2017)
- [Unbiased Learning-to-Rank with Biased Feedback](https://mlsanity.com/paper/1608.04468/) (2016)
- [Evaluation of Retrieval-Augmented Generation: A Survey](https://mlsanity.com/paper/2405.07437/) (2024)
- [MALT: Improving Reasoning with Multi-Agent LLM Training](https://mlsanity.com/paper/2412.01928/) (2024)
- [Optimizing Rank-based Metrics with Blackbox Differentiation](https://mlsanity.com/paper/1912.03500/) (2019)
- [Query Focused Abstractive Summarization: Incorporating Query Relevance, Multi-Document Coverage, and Summary Length Constraints into seq2seq Models](https://mlsanity.com/paper/1801.07704/) (2018)
- [Table-Critic: A Multi-Agent Framework for Collaborative Criticism and Refinement in Table Reasoning](https://mlsanity.com/paper/2502.11799/) (2025)
- [Reinforce LLM Reasoning through Multi-Agent Reflection](https://mlsanity.com/paper/2506.08379/) (2025)
- [Generate Natural Language Explanations for Recommendation](https://mlsanity.com/paper/2101.03392/) (2021)
- [Writing Style Matters: An Examination of Bias and Fairness in Information Retrieval Systems](https://mlsanity.com/paper/2411.13173/) (2024)
- [The Future is Agentic: Definitions, Perspectives, and Open Challenges of Multi-Agent Recommender Systems](https://mlsanity.com/paper/2507.02097/) (2025)
- [Recommendation with Generative Models](https://mlsanity.com/paper/2409.15173/) (2024)
- [ARAG: Agentic Retrieval Augmented Generation for Personalized Recommendation](https://mlsanity.com/paper/2506.21931/) (2025)
- [Abstractive Snippet Generation](https://mlsanity.com/paper/2002.10782/) (2020)
- [Learning to Rank in the Position Based Model with Bandit Feedback](https://mlsanity.com/paper/2004.13106/) (2020)
- [Learning to Generate Text in Arbitrary Writing Styles](https://mlsanity.com/paper/2312.17242/) (2023)
- [MAMM-Refine: A Recipe for Improving Faithfulness in Generation with Multi-Agent Collaboration](https://mlsanity.com/paper/2503.15272/) (2025)
- [AttSum: Joint Learning of Focusing and Summarization with Neural Attention](https://mlsanity.com/paper/1604.00125/) (2016)
- [Solving Black-Box Optimization Challenge via Learning Search Space Partition for Local Bayesian Optimization](https://mlsanity.com/paper/2012.10335/) (2020)
- [CAL-RAG: Retrieval-Augmented Multi-Agent Generation for Content-Aware Layout Design](https://mlsanity.com/paper/2506.21934/) (2025)
- [CARTS: Collaborative Agents for Recommendation Textual Summarization](https://mlsanity.com/paper/2506.17765/) (2025)
- [DRE: Generating Recommendation Explanations by Aligning Large Language Models at Data-level](https://mlsanity.com/paper/2404.06311/) (2024)
- [The Anatomy of Relevance: Topical, Snippet and Perceived Relevance in Search Result Evaluation](https://mlsanity.com/paper/1501.06412/) (2015)
- [Micro-Browsing Models for Search Snippets](https://mlsanity.com/paper/1810.08223/) (2018)

## Abstract (arXiv)

Meta titles and descriptions strongly shape engagement in search and recommendation platforms, yet optimizing them remains challenging. Search engine ranking models are black box environments, explicit labels are unavailable, and feedback such as click-through rate (CTR) arrives only post-deployment. Existing template, LLM, and retrieval-augmented approaches either lack diversity, hallucinate attributes, or ignore whether candidate phrasing has historically succeeded in ranking. This leaves a gap in directly leveraging implicit signals from observable outcomes. We introduce MetaSynth, a multi-agent retrieval-augmented generation framework that learns from implicit search feedback. MetaSynth builds an exemplar library from top-ranked results, generates candidate snippets conditioned on both product content and exemplars, and iteratively refines outputs via evaluator-generator loops that enforce relevance, promotional strength, and compliance. On both proprietary e-commerce data and the Amazon Reviews corpus, MetaSynth outperforms strong baselines across NDCG, MRR, and rank metrics. Large-scale A/B tests further demonstrate 10.26% CTR and 7.51% clicks. Beyond metadata, this work contributes a general paradigm for optimizing content in black-box systems using implicit signals.