mlsanityData to 14 Oct 2026

On Problems of Implicit Context Compression for Software Engineering Agents

arXiv 2605.11051 ↗First posted 11 May 2026cs.SE cs.AI cs.CLcited by 0

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

LLM-based Software Engineering agents face a critical bottleneck: context length limitations cause failures on complex, long-horizon tasks. One promising solution is to encode context as continuous embeddings rather than discrete tokens, enabling denser information storage. We apply the recently proposed In-Context Autoencoder for this purpose. While the method performs well on single-shot common-knowledge and code-understanding tasks, our experiments demonstrate that it fails on multi-step agentic coding tasks. In this paper, we explore this phenomenon and discuss possible factors contributing to this failure.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 30 of its 34 references that are in the map. First posted on arXiv 11 May 2026.

Builds on

30 of 34 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025SWE-smith: Scaling Data for Software Engineering Agents30 Apr
  2. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering6 May

Other references · 28

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  5. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  6. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  7. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  8. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  9. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  10. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  11. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 797
  12. 2021Perceiver IO: A General Architecture for Structured Inputs & Outputscited by 629
16 more
  1. 2024LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compressioncited by 348
  2. 2023The Impact of Positional Encoding on Length Generalization in Transformerscited by 327
  3. 2025CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillationcited by 225
  4. 2022Recurrent Memory Transformercited by 203
  5. 2024Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attentioncited by 200
  6. 2023In-context Autoencoder for Context Compression in a Large Language Modelcited by 199
  7. 2023Text Embeddings Reveal (Almost) As Much As Textcited by 192
  8. 2024RedPajama: an Open Dataset for Training Large Language Modelscited by 182
  9. 2025ACON: Optimizing Context Compression for Long-horizon LLM Agentscited by 136
  10. 2025SoftCoT: Soft Chain-of-Thought for Efficient Reasoning with LLMscited by 116
  11. 2025Look Back to Reason Forward: Revisitable Memory for Long-Context LLM Agentscited by 38
  12. 2024RepoQA: Evaluating Long Context Code Understandingcited by 35
  13. 2025Extending the Context of Pretrained LLMs by Dropping Their Positional Embeddingscited by 26
  14. 2024Natural Is The Best: Model-Agnostic Code Simplification for Pre-trained Large Language Modelscited by 10
  15. 2024Position IDs Matter: An Enhanced Position Layout for Efficient Context Compression in Large Language Modelscited by 9
  16. 2025Solving Context Window Overflow in AI Agentscited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026SWE Context Bench: A Benchmark for Context Learning in Coding
  2. 2026Compress What You See, Not What You Say: Anchored Context Distillation for Latent-Observation Software Engineering Agents
  3. 2026AttnCompress: Dynamic Attention-Guided Trajectory Compression for Software Engineering Agents
  4. 2026SWE-Pruner: Self-Adaptive Context Pruning for Coding Agents
  5. 2026AutoCompact: Learning When to Compact Context in Long-Horizon Coding Agents
  6. 2025Context Engineering for Multi-Agent LLM Code Assistants Using Elicit, NotebookLM, ChatGPT, and Claude Code
  7. 2025LoCoBench-Agent: An Interactive Benchmark for LLM Agents in Long-Context Software Engineering
  8. 2026Yet Even Less Is Even Better For Agentic, Reasoning, and Coding LLMs