# Beyond the Comfort Zone: Emerging Solutions to Overcome Challenges in Integrating LLMs into Software Products

arXiv: https://arxiv.org/abs/2410.12071 · HTML: https://mlsanity.com/paper/2410.12071/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 71 references that are in the map.
- First posted on arXiv 15 Oct 2024.

## Builds on (most cited first, 30 of 31)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting](https://mlsanity.com/paper/2310.11324/) (2023)
- [Automatic Prompt Optimization with "Gradient Descent" and Beam Search](https://mlsanity.com/paper/2305.03495/) (2023)
- [A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications](https://mlsanity.com/paper/2402.07927/) (2024)
- [NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails](https://mlsanity.com/paper/2310.10501/) (2023)
- [AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts](https://mlsanity.com/paper/2010.15980/) (2020)
- [Design Guidelines for Prompt Engineering Text-to-Image Generative Models](https://mlsanity.com/paper/2109.06977/) (2021)
- [Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences](https://mlsanity.com/paper/2404.12272/) (2024)
- [PromptChainer: Chaining Large Language Model Prompts through Visual Programming](https://mlsanity.com/paper/2203.06566/) (2022)
- [Unleashing the potential of prompt engineering for large language models](https://mlsanity.com/paper/2310.14735/) (2023)
- [EvalLM: Interactive Evaluation of Large Language Model Prompts on User-Defined Criteria](https://mlsanity.com/paper/2309.13633/) (2023)
- [Interactive and Visual Prompt Engineering for Ad-hoc Task Adaptation with Large Language Models](https://mlsanity.com/paper/2208.07852/) (2022)
- [Evaluating LLMs at Detecting Errors in LLM Responses](https://mlsanity.com/paper/2404.03602/) (2024)
- [A Survey of Prompt Engineering Methods in Large Language Models for Different NLP Tasks](https://mlsanity.com/paper/2407.12994/) (2024)
- [Where Responsible AI meets Reality: Practitioner Perspectives on Enablers for shifting Organizational Practices](https://mlsanity.com/paper/2006.12358/) (2020)
- [Evaluating the Evaluator: Measuring LLMs' Adherence to Task Evaluation Instructions](https://mlsanity.com/paper/2408.08781/) (2024)
- [ConstitutionMaker: Interactively Critiquing Large Language Models by Converting Feedback into Principles](https://mlsanity.com/paper/2310.15428/) (2023)
- [Farsight: Fostering Responsible AI Awareness During AI Application Prototyping](https://mlsanity.com/paper/2402.15350/) (2024)
- [Artificial Intelligence Index Report 2024](https://mlsanity.com/paper/2405.19522/) (2024)
- [(Why) Is My Prompt Getting Worse? Rethinking Regression Testing for Evolving LLM APIs](https://mlsanity.com/paper/2311.11123/) (2023)
- [A Meta-Summary of Challenges in Building Products with ML Components -- Collecting Experiences from 4758+ Practitioners](https://mlsanity.com/paper/2304.00078/) (2023)
- [ScatterShot: Interactive In-context Example Curation for Text Transformation](https://mlsanity.com/paper/2302.07346/) (2023)
- [PromptAid: Prompt Exploration, Perturbation, Testing and Iteration using Visual Analytics for Large Language Models](https://mlsanity.com/paper/2304.01964/) (2023)
- [SPADE: Synthesizing Data Quality Assertions for Large Language Model Pipelines](https://mlsanity.com/paper/2401.03038/) (2024)
- [Prompts Are Programs Too! Understanding How Developers Build Software Containing Prompts](https://mlsanity.com/paper/2409.12447/) (2024)
- [Building Your Own Product Copilot: Challenges, Opportunities, and Needs](https://mlsanity.com/paper/2312.14231/) (2023)
- [Rethinking Software Engineering in the Foundation Model Era: From Task-Driven AI Copilots to Goal-Driven AI Pair Programmers](https://mlsanity.com/paper/2404.10225/) (2024)
- [Development in times of hype: How freelancers explore Generative AI?](https://mlsanity.com/paper/2401.05790/) (2024)
- [Visualizing Linguistic Diversity of Text Datasets Synthesized by Large Language Models](https://mlsanity.com/paper/2305.11364/) (2023)

## Built on by (most cited first, 11 of 11)

- [A Large-Scale Evolvable Dataset for Model Context Protocol Ecosystem and Security Analysis](https://mlsanity.com/paper/2506.23474/) (2025)
- [LLM Company Policies and Policy Implications in Software Organizations](https://mlsanity.com/paper/2510.06718/) (2025)
- [Agentic Pipelines in Embedded Software Engineering: Emerging Practices and Challenges](https://mlsanity.com/paper/2601.10220/) (2026)
- [PrefPO: Pairwise Preference Prompt Optimization](https://mlsanity.com/paper/2603.19311/) (2026)
- [How Do Practitioners Build SE Agents? Insights from a Mixed-Methods Study](https://mlsanity.com/paper/2607.10856/) (2026)
- [A Research Roadmap for Augmenting Software Engineering Processes and Software Products with Generative AI](https://mlsanity.com/paper/2510.26275/) (2025)
- [Key Considerations for Domain Expert Involvement in LLM Design and Evaluation: An Ethnographic Study](https://mlsanity.com/paper/2602.14357/) (2026)
- [The Hitchhikers Guide to Production-ready Trustworthy Foundation Model powered Software (FMware)](https://mlsanity.com/paper/2505.10640/) (2025)
- [Results-Actionability Gap: Understanding How Practitioners Evaluate LLM Products in the Wild](https://mlsanity.com/paper/2604.16304/) (2026)
- [Beyond the Org Chart: AI and the Transformation of Invisible Work](https://mlsanity.com/paper/2605.22707/) (2026)
- [A Framework for Generating Valid Context-Specific Benchmarks through Expert Guidance](https://mlsanity.com/paper/2609.16592/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) are increasingly embedded into software products across diverse industries, enhancing user experiences, but at the same time introducing numerous challenges for developers. Unique characteristics of LLMs force developers, who are accustomed to traditional software development and evaluation, out of their comfort zones as the LLM components shatter standard assumptions about software systems. This study explores the emerging solutions that software developers are adopting to navigate the encountered challenges. Leveraging a mixed-method research, including 26 interviews and a survey with 332 responses, the study identifies 19 emerging solutions regarding quality assurance that practitioners across several product teams at Microsoft are exploring. The findings provide valuable insights that can guide the development and evaluation of LLM-based products more broadly in the face of these challenges.