Do Influence Tactics Matter? Investigating Prompt Framing Effects in LLM Code Generation
Not on the map. Why?
Abstract (arXiv)
Large Language Models (LLMs) are increasingly integrated into software engineering workflows, helping developers write, debug, test, and maintain code. While prompt wording and structure are known to influence model performance, the impact of psychologically inspired prompt framings remains unexplored. This study investigates whether different psychology-based communication strategies that humans use to persuade or motivate others can lead to more effective prompt framing, which may, in turn, affect LLM behaviour in coding tasks. Drawing on Yukl & Falbe's well-known taxonomy, we operationalized eight influence tactics (like rational persuasion, ingratiation, and exchange) into reproducible prompt templates. These prompt templates were evaluated across five leading open-weight LLMs using two widely adopted benchmarks: LiveCodeBench and SWE-bench Verified. We assessed the resulting code output on four key software quality dimensions: functional correctness, quality, maintainability, and security. Our results show that certain influence-induced prompt framings, particularly those emphasizing urgency, were associated with reduced correctness and security. This work presents the first large-scale empirical study of influence-induced prompt framing in software engineering tasks, offering insights into how linguistic cues may shape LLM outputs. We conclude with practical insights for designing transparent and interpretable human-AI interactions in code generation.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 40 of its 81 references that are in the map. First posted on arXiv 11 Aug 2026.
Builds on
40 of 81 references.
References · 40
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2025Qwen3 Technical Reportcited by 8,195
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
- 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
- 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
- 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 684
- 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 628
28 more
- 2023A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPTcited by 612
- 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 554
- 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 551
- 2021PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Modelscited by 402
- 2024ProSA: Assessing and Understanding the Prompt Sensitivity of LLMscited by 145
- 2023Large Language Models Understand and Can be Enhanced by Emotional Stimulicited by 143
- 2024Debug like a Human: A Large Language Model Debugger via Verifying Runtime Execution Step-by-stepcited by 123
- 2025Large Language Models Often Know When They Are Being Evaluatedcited by 105
- 2023Can ChatGPT replace StackOverflow? A Study on Robustness and Reliability of Large Language Model Code Generationcited by 72
- 2023Principled Instructions Are All You Need for Questioning LLaMA-1/2, GPT-3.5/4cited by 48
- 2024Should We Respect LLMs? A Cross-Lingual Study on the Influence of Prompt Politeness on LLM Performancecited by 41
- 2024Prompting Techniques for Secure Code Generation: A Systematic Investigationcited by 33
- 2023Bounding the Capabilities of Large Language Models in Open Text Generation with Prompt Constraintscited by 31
- 2023Improving ChatGPT Prompt for Code Generationcited by 28
- 2025Guidelines for Empirical Studies in Software Engineering involving Large Language Modelscited by 18
- 2024Can We Trust Large Language Models Generated Code? A Framework for In-Context Learning, Security Patterns, and Code Evaluations Across Diverse LLMscited by 17
- 2024Evaluating Software Development Agents: Patch Patterns, Code Quality, and Issue Complexity in Real-World GitHub Scenarioscited by 16
- 2025Large Language Models for Code Generation: The Practitioners Perspectivecited by 9
- 2025Prompt Sentiment: The Catalyst for LLM Changecited by 9
- 2019Time Pressure in Software Engineering: A Systematic Reviewcited by 8
- 2024Analyzing Prompt Influence on Automated Method Generation: An Empirical Study with Copilotcited by 6
- 2025Large Language Models for Unit Testing: A Systematic Literature Reviewcited by 6
- 2025Developer-LLM Conversations: An Empirical Study of Interactions and Generated Code Qualitycited by 6
- 2024Automated Prompt Engineering for Cost-Effective Code Generation Using Evolutionary Algorithmcited by 5
- 2024Understanding Defects in Generated Codes by Language Modelscited by 4
- 2024Examination of Code generated by Large Language Modelscited by 3
- 2025LRASGen: LLM-based RESTful API Specification Generationcited by 3
- 2026Framing Effects in Independent-Agent Large Language Models: A Cross-Family Behavioral Analysiscited by 1
Built on by
Not cited yet.