mlsanityData to 14 Oct 2026

Enhancing Mobile "How-to" Queries with Automated Search Results Verification and Reranking

arXiv 2404.08860 ↗First posted 13 Apr 2024cs.IR cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Many people use search engines to find online guidance to solve computer or mobile device problems. Users frequently encounter challenges in identifying effective solutions from search results, often wasting time trying ineffective solutions that seem relevant yet fail to solve real problems. This paper introduces a novel approach to improving the accuracy and relevance of online technical support search results through automated search results verification and reranking. Taking "How-to" queries specific to on-device execution as a starting point, we developed the first solution that allows an AI agent to interpret and execute step-by-step instructions in the search results in a controlled Android environment. We further integrated the agent's findings into a reranking mechanism that orders search results based on the success indicators of the tested solutions. The paper details the architecture of our solution and a comprehensive evaluation of the system through a series of tests across various application domains. The results demonstrate a significant improvement in the quality and reliability of the top-ranked results. Our findings suggest a paradigm shift in how search engine ranking for online technical support help can be optimized, offering a scalable and automated solution to the pervasive challenge of finding effective and reliable online help.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 34 of its 67 references that are in the map. First posted on arXiv 13 Apr 2024.

Builds on

34 of 67 references.

References · 34

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  4. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  5. 2017Billion-scale similarity search with GPUscited by 2,497
  6. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  7. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,319
  8. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,035
  9. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 681
  10. 2022LayoutLMv3: Pre-training for Document AI with Unified Text and Image Maskingcited by 485
  11. 2023AppAgent: Multimodal Agents as Smartphone Userscited by 415
  12. 2023Augmented Language Models: a Surveycited by 403
22 more
  1. 2023Android in the Wild: A Large-Scale Dataset for Android Device Controlcited by 319
  2. 2022Pre-Trained Language Models for Interactive Decision-Makingcited by 221
  3. 2023You Only Look at Screens: Multimodal Chain-of-Action Agentscited by 185
  4. 2020Mapping Natural Language Instructions to Mobile UI Action Sequencescited by 181
  5. 2023GPT-4V in Wonderland: Large Multimodal Models for Zero-Shot Smartphone GUI Navigationcited by 137
  6. 2017A Deep Relevance Matching Model for Ad-hoc Retrievalcited by 126
  7. 2013Introducing LETOR 4.0 Datasetscited by 121
  8. 2019Stochastic Optimization of Sorting Networks via Continuous Relaxationscited by 111
  9. 2024LayoutLLM: Layout Instruction Tuning with Large Language Models for Document Understandingcited by 95
  10. 2021AndroidEnv: A Reinforcement Learning Platform for Androidcited by 90
  11. 2022META-GUI: Towards Multi-modal Conversational Agents on Mobile GUIcited by 81
  12. 2016Learning to Match Using Local and Distributed Representations of Text for Web Searchcited by 68
  13. 2021MarkupLM: Pre-training of Text and Markup Language for Visually-rich Document Understandingcited by 45
  14. 2021NeuralNDCG: Direct Optimisation of a Ranking Metric via Differentiable Relaxation of Sortingcited by 41
  15. 2021GPT3-to-plan: Extracting plans from text using GPT-3cited by 26
  16. 2022UGIF: UI Grounded Instruction Followingcited by 25
  17. 2022DOM-LM: Learning Generalizable Representations for HTML Documentscited by 24
  18. 2020Context-Aware Learning to Rank with Self-Attentioncited by 22
  19. 2022XDoc: Unified Pre-training for Cross-Format Document Understandingcited by 12
  20. 2018Web2Text: Deep Structured Boilerplate Removalcited by 10
  21. 2022PLAtE: A Large-scale Dataset for List Page Web Extractioncited by 3
  22. 2021Don't read, just look: Main content extraction from web pages using visual featurescited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 0; 2026: 1.

3 papers.

  1. 2024Large Language Model-Brained GUI Agents: A Surveycited by 162
  2. 2024Read Anywhere Pointed: Layout-aware GUI Screen Reading with Tree-of-Lens Groundingcited by 18
  3. 2026Software Engineering for and with GUI Agentcited by 1