Lost in Context Compaction: When LLM Systems Drop Side Constraints

When the context window is under pressure, LLM systems compact prior context to continue ongoing tasks. We study Side Constraints (SCs), user-issued instructions such as "do not delete any emails until I confirm," that constrain an LLM's behavior across subsequent turns but can be silently dropped during compaction. In an inbox experiment, an agent deleted emails without approval in 30/100 sessions after compaction, compared with none before compaction. To systematically evaluate SC preservation, we introduce CONPACT, an evaluation suite measuring semantic retention and downstream compliance across three scenarios: multi-turn chat, multi-turn tool use, and long-horizon research. The evaluated compactors retain only 17% of injected SCs on average. Loss occurs across compactors and settings, with retention varying by compactor, context length, SC framing, and injection location. We propose an SC extractor that maintains constraints separately from the task summary and appends them to the compacted context, achieving over 90% retention across all three scenarios without modifying the compactor or LLM. Code and data are available at https://github.com/ZhiqiEliWang/compaction-integrity.