mlsanityData to 14 Oct 2026

ASSURE: Metamorphic Testing for AI-powered Browser Extensions

arXiv 2507.05307 ↗First posted 7 Jul 2025cs.SE cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

The integration of Large Language Models (LLMs) into browser extensions has revolutionized web browsing, enabling sophisticated functionalities like content summarization, intelligent translation, and context-aware writing assistance. However, these AI-powered extensions introduce unprecedented challenges in testing and reliability assurance. Traditional browser extension testing approaches fail to address the non-deterministic behavior, context-sensitivity, and complex web environment integration inherent to LLM-powered extensions. Similarly, existing LLM testing methodologies operate in isolation from browser-specific contexts, creating a critical gap in effective evaluation frameworks. To bridge this gap, we present ASSURE, a modular automated testing framework specifically designed for AI-powered browser extensions. ASSURE comprises three principal components: (1) a modular test case generation engine that supports plugin-based extension of testing scenarios, (2) an automated execution framework that orchestrates the complex interactions between web content, extension processing, and AI model behavior, and (3) a configurable validation pipeline that systematically evaluates behavioral consistency and security invariants rather than relying on exact output matching. Our evaluation across six widely-used AI browser extensions demonstrates ASSURE's effectiveness, identifying 531 distinct issues spanning security vulnerabilities, metamorphic relation violations, and content alignment problems. ASSURE achieves 6.4x improved testing throughput compared to manual approaches, detecting critical security vulnerabilities within 12.4 minutes on average. This efficiency makes ASSURE practical for integration into development pipelines, offering a comprehensive solution to the unique challenges of testing AI-powered browser extensions.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 13 of its 42 references that are in the map. First posted on arXiv 7 Jul 2025.

Builds on

13 of 42 references.

References · 13

  1. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  2. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 1,006
  3. 2023Prompt Injection attack against LLM-integrated Applicationscited by 605
  4. 2023Multi-step Jailbreaking Privacy Attacks on ChatGPTcited by 333
  5. 2023PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimizationcited by 316
  6. 2017DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Carscited by 251
  7. 2023PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Promptscited by 226
  8. 2024EIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakagecited by 150
  9. 2023Revisiting Out-of-distribution Robustness in NLP: Benchmark, Analysis, and LLMs Evaluationscited by 97
  10. 2020Metamorphic Testing: A New Approach for Generating Next Test Casescited by 75
  11. 2022GLUE-X: Evaluating Natural Language Understanding Models from an Out-of-distribution Generalization Perspectivecited by 72
  12. 2020A Declarative Metamorphic Testing Framework for Autonomous Drivingcited by 10
1 more
  1. 2022Metamorphic Testing for Web System Securitycited by 1

Built on by

Not cited yet.