Large Language Models Do Not Always Need Readable Language
Not on the map. Why?
Abstract (arXiv)
Large language models (LLMs) are commonly prompted and interfaced with human-readable natural language, even when the intended reader is another model. This paper investigates whether semantic information can be encoded in compact, non-standard textual forms that sacrifice human readability while remaining recoverable by LLMs. We refer to this class of model-centric textual representations as BabelTele, approached here not as a fixed protocol but as an empirical probe into LLMs' capacity to generate and interpret such representations. Through readability diagnostics, model likelihood measures, human questionnaires, and downstream task evaluations, we find that BabelTele can substantially depart from ordinary natural language while preserving core semantics for instruction-tuned LLMs. As a task-agnostic representational paradigm, BabelTele demonstrates high information density, maintaining 99.5% semantic fidelity even when the text volume is condensed to 27.9% of its original length. We further evaluate its semantic robustness in cross-model transfer, agent memory, and multi-agent communication. Results suggest that BabelTele can reduce context overhead while generally maintaining reliable downstream performance, although its effectiveness depends on the compressor-reader pair and task setting. These findings indicate that human readability, natural-language typicality, and model-side semantic recoverability can be partially decoupled, opening a path toward model-native representations in future exploration of LLM systems.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 42 of its 59 references that are in the map. First posted on arXiv 18 Jun 2026.
Builds on
42 of 59 references.
References · 42
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
- 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
- 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
- 2025A-MEM: Agentic Memory for LLM Agentscited by 1,063
- 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 1,040
- 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 830
30 more
- 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 591
- 2026GLM-5: from Vibe Coding to Agentic Engineeringcited by 587
- 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 575
- 2026Kimi K2.5: Visual Agentic Intelligencecited by 501
- 2023LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compressioncited by 453
- 2023Learning to Compress Prompts with Gist Tokenscited by 390
- 2024LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitaskscited by 379
- 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Contextcited by 369
- 2023Adapting Language Models to Compress Contextscited by 356
- 2023StructGPT: A General Framework for Large Language Model to Reason over Structured Datacited by 343
- 2017Emergence of Grounded Compositional Language in Multi-Agent Populationscited by 263
- 2025DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agentscited by 248
- 2025MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agentcited by 246
- 2021QuALITY: Question Answering with Long Input Texts, Yes!cited by 208
- 2017Emergence of Language with Multi-agent Games: Learning to Communicate with Sequences of Symbolscited by 125
- 2024Long Context Compression with Activation Beaconcited by 104
- 2023MeetingBank: A Benchmark Dataset for Meeting Summarizationcited by 84
- 2023Exchange-of-Thought: Enhancing Large Language Model Capabilities through Cross-Model Communicationcited by 82
- 2024Learning to Compress Prompt in Natural Language Formatscited by 55
- 2024A Scalable Communication Protocol for Networks of Large Language Modelscited by 45
- 2025Enabling Agents to Communicate Entirely in Latent Spacecited by 34
- 2024Symbolic Prompt Program Search: A Structure-Aware Approach to Efficient Compile-Time Prompt Optimizationcited by 30
- 2025Communicating Activations Between Language Model Agentscited by 16
- 2026Latent Context Compilation: Distilling Long Context into Compact Portable Memorycited by 13
- 2026Memori: A Persistent Memory Layer for Efficient, Context-Aware LLM Agentscited by 12
- 2025Enhancing RAG Efficiency with Adaptive Context Compressioncited by 8
- 2024Enhancing and Accelerating Large Language Models via Instruction-Aware Contextual Compressioncited by 4
- 2025ECoRAG: Evidentiality-guided Compression for Long Context RAGcited by 4
- 2025ILRe: Intermediate Layer Retrieval for Context Compression in Causal Language Modelscited by 3
- 2026Semantic Compression of LLM Instructions via Symbolic Metalanguagescited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
- 2026On Language Drift during RLVR Post-Trainingcited by 0
- 2026On Language Drift during RLVR Post-Training1 Oct 2026
Similar, not cited
Close in wording, no citation either way
- 2025Representation Engineering for Large-Language Models: Survey and Research Challenges
- 2024Towards Uncovering How Large Language Model Works: An Explainability Perspective
- 2025Explainability of Large Language Models: Opportunities and Challenges toward Generating Trustworthy Explanations
- 2026Towards Intrinsic Interpretability of Large Language Models:A Survey of Design Principles and Architectures
- 2025Direct Semantic Communication Between Large Language Models via Vector Translation
- 2026Is Human-Readable Text Necessary for Effective LLM Fine-Tuning?
- 2024Improving Large Language Model (LLM) fidelity through context-aware grounding: A systematic approach to reliability and veracity
- 2026Latent Communication Between Language Model Agents: Channels, Alignment, and the Limits of Text