# Interpreting User Requests in the Context of Natural Language Standing Instructions

arXiv: https://arxiv.org/abs/2311.09796 · HTML: https://mlsanity.com/paper/2311.09796/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 50 references that are in the map.
- First posted on arXiv 16 Nov 2023.

## Builds on (most cited first, 30 of 31)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://mlsanity.com/paper/2307.16789/) (2023)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [Unsupervised Dense Information Retrieval with Contrastive Learning](https://mlsanity.com/paper/2112.09118/) (2021)
- [CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation](https://mlsanity.com/paper/2109.00859/) (2021)
- [Personalizing Dialogue Agents: I have a dog, do you have pets too?](https://mlsanity.com/paper/1801.07243/) (2018)
- [A Survey on In-context Learning](https://mlsanity.com/paper/2301.00234/) (2022)
- [Learning To Retrieve Prompts for In-Context Learning](https://mlsanity.com/paper/2112.08633/) (2021)
- [LaMP: When Large Language Models Meet Personalization](https://mlsanity.com/paper/2304.11406/) (2023)
- [Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Dataset](https://mlsanity.com/paper/1909.05855/) (2019)
- [Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignment](https://mlsanity.com/paper/2308.05374/) (2023)
- [Memory Networks](https://mlsanity.com/paper/1410.3916/) (2014)
- [Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks](https://mlsanity.com/paper/1502.05698/) (2015)
- [A Persona-Based Neural Conversation Model](https://mlsanity.com/paper/1603.06155/) (2016)
- [SatLM: Satisfiability-Aided Language Models Using Declarative Prompting](https://mlsanity.com/paper/2305.09656/) (2023)
- [CLIN: A Continually Learning Language Agent for Rapid Task Adaptation and Generalization](https://mlsanity.com/paper/2310.10134/) (2023)
- [Answering Questions by Meta-Reasoning over Multiple Chains of Thought](https://mlsanity.com/paper/2304.13007/) (2023)
- [Like hiking? You probably enjoy nature: Persona-grounded Dialog with Commonsense Expansions](https://mlsanity.com/paper/2010.03205/) (2020)
- [ChatGPT for Zero-shot Dialogue State Tracking: A Solution or an Opportunity?](https://mlsanity.com/paper/2306.01386/) (2023)
- [Learning to Repair: Repairing model output errors after deployment using a dynamic memory of feedback](https://mlsanity.com/paper/2112.09737/) (2021)
- [Personalization in Goal-Oriented Dialog](https://mlsanity.com/paper/1706.07503/) (2017)
- [Knowledge-Enhanced Personalized Review Generation with Capsule Graph Neural Network](https://mlsanity.com/paper/2010.01480/) (2020)
- [Towards Fair Evaluation of Dialogue State Tracking by Flexible Incorporation of Turn-level Performances](https://mlsanity.com/paper/2204.03375/) (2022)
- [BenchCLAMP: A Benchmark for Evaluating Language Models on Syntactic and Semantic Parsing](https://mlsanity.com/paper/2206.10668/) (2022)
- [Learning to Learn Semantic Parsers from Natural Language Supervision](https://mlsanity.com/paper/1902.08373/) (2019)
- [Ontologically Faithful Generation of Non-Player Character Dialogues](https://mlsanity.com/paper/2212.10618/) (2022)

## Built on by (most cited first, 5 of 5)

- [ToolDial: Multi-turn Dialogue Generation Method for Tool-Augmented Language Models](https://mlsanity.com/paper/2503.00564/) (2025)
- [Sample-Efficient Online Learning in LM Agents via Hindsight Trajectory Rewriting](https://mlsanity.com/paper/2510.10304/) (2025)
- [What Makes a Good Natural Language Prompt?](https://mlsanity.com/paper/2506.06950/) (2025)
- [Latent Preference Modeling for Multi-Session Personalized Tool Calling](https://mlsanity.com/paper/2604.17886/) (2026)
- [How Strongly Should Task State Influence an LLM Agent?](https://mlsanity.com/paper/2609.25686/) (2026)

## Abstract (arXiv)

Users of natural language interfaces, generally powered by Large Language Models (LLMs),often must repeat their preferences each time they make a similar request. We describe an approach to LLM-based dialogue modeling in which persistent user constraints and preferences -- collectively termed standing instructions -- as additional context for such interfaces. For example, when a user states "I'm hungry", a previously expressed preference for Persian food can be automatically added to the LLM prompt, influencing the search for relevant restaurants. We develop NLSI, a language-to-program dataset consisting of over 2.4K dialogues spanning 17 domains, where each dialogue is paired with a user profile (a set of users specific standing instructions) and corresponding structured representations (API calls). A key challenge in NLSI is to identify which subset of the standing instructions is applicable to a given dialogue. NLSI contains diverse phenomena, from simple preferences to interdependent instructions such as triggering a hotel search whenever the user is booking tickets to an event. We conduct experiments on NLSI using prompting with large language models and various retrieval approaches, achieving a maximum of 44.7% exact match on API prediction. Our results demonstrate the challenges in identifying the relevant standing instructions and their interpretation into API calls.