mlsanityData to 7 Oct 2026

MetaSynth: Multi-Agent Metadata Generation from Implicit Feedback in Black-Box Systems

arXiv 2510.01523 ↗First posted 1 Oct 2025cs.IRcited by 0

On the map: Retrieval-augmented generation → Agentic RAG, agents and memory

also in 1 more
Abstract (arXiv)

Meta titles and descriptions strongly shape engagement in search and recommendation platforms, yet optimizing them remains challenging. Search engine ranking models are black box environments, explicit labels are unavailable, and feedback such as click-through rate (CTR) arrives only post-deployment. Existing template, LLM, and retrieval-augmented approaches either lack diversity, hallucinate attributes, or ignore whether candidate phrasing has historically succeeded in ranking. This leaves a gap in directly leveraging implicit signals from observable outcomes. We introduce MetaSynth, a multi-agent retrieval-augmented generation framework that learns from implicit search feedback. MetaSynth builds an exemplar library from top-ranked results, generates candidate snippets conditioned on both product content and exemplars, and iteratively refines outputs via evaluator-generator loops that enforce relevance, promotional strength, and compliance. On both proprietary e-commerce data and the Amazon Reviews corpus, MetaSynth outperforms strong baselines across NDCG, MRR, and rank metrics. Large-scale A/B tests further demonstrate 10.26% CTR and 7.51% clicks. Beyond metadata, this work contributes a general paradigm for optimizing content in black-box systems using implicit signals.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Agentic RAG, agents and memory, one of 15 directions in Retrieval-augmented generation. Builds on 33 of its 42 references that are in the map. First posted on arXiv 1 Oct 2025.

Builds on

33 of 42 references.

References · 33

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
  5. 2020Bias and Debias in Recommender System: A Survey and Future Directionscited by 239
  6. 2024Bridging Language and Items for Retrieval and Recommendation: Benchmarking LLMs as Semantic Encoderscited by 213
  7. 2017Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluationcited by 206
  8. 2016Unbiased Learning-to-Rank with Biased Feedbackcited by 155
  9. 2024Evaluation of Retrieval-Augmented Generation: A Surveycited by 146
  10. 2024MALT: Improving Reasoning with Multi-Agent LLM Trainingcited by 71
  11. 2019Optimizing Rank-based Metrics with Blackbox Differentiationcited by 58
  12. 2018Query Focused Abstractive Summarization: Incorporating Query Relevance, Multi-Document Coverage, and Summary Length Constraints into seq2seq Modelscited by 39
21 more
  1. 2025Table-Critic: A Multi-Agent Framework for Collaborative Criticism and Refinement in Table Reasoningcited by 32
  2. 2025Reinforce LLM Reasoning through Multi-Agent Reflectioncited by 26
  3. 2021Generate Natural Language Explanations for Recommendationcited by 22
  4. 2024Writing Style Matters: An Examination of Bias and Fairness in Information Retrieval Systemscited by 15
  5. 2025The Future is Agentic: Definitions, Perspectives, and Open Challenges of Multi-Agent Recommender Systemscited by 14
  6. 2024Recommendation with Generative Modelscited by 13
  7. 2025ARAG: Agentic Retrieval Augmented Generation for Personalized Recommendationcited by 12
  8. 2020Abstractive Snippet Generationcited by 11
  9. 2020Learning to Rank in the Position Based Model with Bandit Feedbackcited by 10
  10. 2023Learning to Generate Text in Arbitrary Writing Stylescited by 9
  11. 2025MAMM-Refine: A Recipe for Improving Faithfulness in Generation with Multi-Agent Collaborationcited by 9
  12. 2016AttSum: Joint Learning of Focusing and Summarization with Neural Attentioncited by 7
  13. 2020Solving Black-Box Optimization Challenge via Learning Search Space Partition for Local Bayesian Optimizationcited by 7
  14. 2025CAL-RAG: Retrieval-Augmented Multi-Agent Generation for Content-Aware Layout Designcited by 7
  15. 2025CARTS: Collaborative Agents for Recommendation Textual Summarizationcited by 6
  16. 2024DRE: Generating Recommendation Explanations by Aligning Large Language Models at Data-levelcited by 4
  17. 2015The Anatomy of Relevance: Topical, Snippet and Perceived Relevance in Search Result Evaluationcited by 1
  18. 2018Micro-Browsing Models for Search Snippetscited by 1
  19. 2024Mitigate Position Bias with Coupled Ranking Bias on CTR Predictioncited by 1
  20. 2025Aligning Language Models with Observational Data: Opportunities and Risks from a Causal Perspectivecited by 1
  21. 2025A Control Function Framework for Mitigating Position Bias in Learning to Rank Systemscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026QueryAgent-R1: Bridging Query Generation and Product Retrieval for E-Commerce Query Recommendation
  2. 2026RRCM: Ranking-Driven Retrieval over Collaborative and Meta Memories for LLM Recommendation
  3. 2025A Multi-Agent Perspective on Modern Information Retrieval
  4. 2024Learning to Rank for Multiple Retrieval-Augmented Models through Iterative Utility Maximization
  5. 2026Prism-Reranker: Beyond Relevance Scoring -- Jointly Producing Contributions and Evidence for Agentic Retrieval
  6. 2026Learning to Retrieve from Agent Trajectories
  7. 2026MATRAG: Multi-Agent Transparent Retrieval-Augmented Generation for Explainable Recommendations
  8. 2025RankArena: A Unified Platform for Evaluating Retrieval, Reranking and RAG with Human and LLM Feedback