# ReFinE: Streamlining UI Mockup Iteration with Research Findings

arXiv: https://arxiv.org/abs/2604.04353 · HTML: https://mlsanity.com/paper/2604.04353/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 48 references that are in the map.
- First posted on arXiv 6 Apr 2026.

## Builds on (most cited first, 19 of 19)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis](https://mlsanity.com/paper/2307.12856/) (2023)
- [Emergent Analogical Reasoning in Large Language Models](https://mlsanity.com/paper/2212.09196/) (2022)
- [Enabling Conversational Interaction with Mobile UI using Large Language Models](https://mlsanity.com/paper/2209.08655/) (2022)
- [HTLM: Hyper-Text Pre-Training and Prompting of Language Models](https://mlsanity.com/paper/2107.06955/) (2021)
- [MarkupLM: Pre-training of Text and Markup Language for Visually-rich Document Understanding](https://mlsanity.com/paper/2110.08518/) (2021)
- [VINS: Visual Search for Mobile User Interface Design](https://mlsanity.com/paper/2102.05216/) (2021)
- [DOM-LM: Learning Generalizable Representations for HTML Documents](https://mlsanity.com/paper/2201.10608/) (2022)
- [Paper Plain: Making Medical Research Papers Approachable to Healthcare Consumers with Natural Language Processing](https://mlsanity.com/paper/2203.00130/) (2022)
- [Generating Automatic Feedback on UI Mockups with Large Language Models](https://mlsanity.com/paper/2403.13139/) (2024)
- [A Suite of Generative Tasks for Multi-Level Multimodal Webpage Understanding](https://mlsanity.com/paper/2305.03668/) (2023)
- [PaperWeaver: Enriching Topical Paper Alerts by Contextualizing Recommended Papers with User-collected Papers](https://mlsanity.com/paper/2403.02939/) (2024)
- [Leveraging Multimodal LLM for Inspirational User Interface Search](https://mlsanity.com/paper/2501.17799/) (2025)
- [Misty: UI Prototyping Through Interactive Conceptual Blending](https://mlsanity.com/paper/2409.13900/) (2024)
- [Qlarify: Recursively Expandable Abstracts for Directed Information Retrieval over Scientific Papers](https://mlsanity.com/paper/2310.07581/) (2023)
- [Analogy Mining for Specific Design Needs](https://mlsanity.com/paper/1712.06880/) (2017)
- [From Paper to Card: Transforming Design Implications with Generative AI](https://mlsanity.com/paper/2403.08137/) (2024)
- [AI-Assisted Causal Pathway Diagram for Human-Centered Design](https://mlsanity.com/paper/2403.08111/) (2024)
- [DIDUP: Dynamic Iterative Development for UI Prototyping](https://mlsanity.com/paper/2407.08474/) (2024)

## Abstract (arXiv)

Although HCI research papers offer valuable design insights, designers often struggle to apply them in design workflows due to difficulties in finding relevant literature, understanding technical jargon, the lack of contextualization, and limited actionability. To address these challenges, we present ReFinE, a Figma plugin that supports real-time design iteration by surfacing contextualized insights from research papers. ReFinE identifies and synthesizes design implications from HCI literature relevant to the mockup's design context, and tailors this research evidence to a specific design mockup by providing actionable visual guidance on how to update the mockup. To assess the system's effectiveness, we conducted a technical evaluation and a user study. Results show that ReFinE effectively synthesizes and contextualizes design implications, reducing cognitive load and improving designers' ability to integrate research evidence into UI mockups. This work contributes to bridging the gap between research and design practice by presenting a tool for embedding scholarly insights into the UI design process.