# Urban Generative Intelligence (UGI): A Foundational Platform for Agents in Embodied City Environment

arXiv: https://arxiv.org/abs/2312.11813 · HTML: https://mlsanity.com/paper/2312.11813/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use.
- Builds on 61 of its 187 references that are in the map.
- First posted on arXiv 19 Dec 2023.

## Placed in

- LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/)

## Builds on (most cited first, 30 of 61)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [Voyager: An Open-Ended Embodied Agent with Large Language Models](https://mlsanity.com/paper/2305.16291/) (2023)
- [PaLM-E: An Embodied Multimodal Language Model](https://mlsanity.com/paper/2303.03378/) (2023)
- [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2308.00352/) (2023)
- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023)
- [ALFWorld: Aligning Text and Embodied Environments for Interactive Learning](https://mlsanity.com/paper/2010.03768/) (2020)
- [WizardLM: Empowering large pre-trained language models to follow complex instructions](https://mlsanity.com/paper/2304.12244/) (2023)
- [GLM-130B: An Open Bilingual Pre-trained Model](https://mlsanity.com/paper/2210.02414/) (2022)
- [Enhancing Chat Language Models by Scaling High-quality Instructional Conversations](https://mlsanity.com/paper/2305.14233/) (2023)
- [C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Models](https://mlsanity.com/paper/2305.08322/) (2023)
- [Baichuan 2: Open Large-scale Language Models](https://mlsanity.com/paper/2309.10305/) (2023)
- [A Survey on Knowledge Graphs: Representation, Acquisition and Applications](https://mlsanity.com/paper/2002.00388/) (2020)
- [Knowledge Graphs](https://mlsanity.com/paper/2003.02320/) (2020)
- [Structure and Content-Guided Video Synthesis with Diffusion Models](https://mlsanity.com/paper/2302.03011/) (2023)
- [FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU](https://mlsanity.com/paper/2303.06865/) (2023)
- [Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies](https://mlsanity.com/paper/2208.10264/) (2022)
- [CMMLU: Measuring massive multitask language understanding in Chinese](https://mlsanity.com/paper/2306.09212/) (2023)
- [DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale](https://mlsanity.com/paper/2207.00032/) (2022)
- [SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents](https://mlsanity.com/paper/2310.11667/) (2023)

## Built on by (most cited first, 30 of 37)

- [Understanding World or Predicting Future? A Comprehensive Survey of World Models](https://mlsanity.com/paper/2411.14499/) (2024)
- [LLM as a Mastermind: A Survey of Strategic Reasoning with Large Language Models](https://mlsanity.com/paper/2404.01230/) (2024)
- [UniST: A Prompt-Empowered Universal Model for Urban Spatio-Temporal Prediction](https://mlsanity.com/paper/2402.11838/) (2024)
- [A Survey on LLM-based Multi-Agent System: Recent Advances and New Frontiers in Application](https://mlsanity.com/paper/2412.17481/) (2024)
- [Knowledge Graphs Meet Multi-Modal Learning: A Comprehensive Survey](https://mlsanity.com/paper/2402.05391/) (2024)
- [Y Social: an LLM-powered Social Media Digital Twin](https://mlsanity.com/paper/2408.00818/) (2024)
- [CharacterBox: Evaluating the Role-Playing Capabilities of LLMs in Text-Based Virtual Worlds](https://mlsanity.com/paper/2412.05631/) (2024)
- [UrbanVideo-Bench: Benchmarking Vision-Language Models on Embodied Intelligence with Video Data in Urban Spaces](https://mlsanity.com/paper/2503.06157/) (2025)
- [OpenCity: A Scalable Platform to Simulate Urban Activities with Massive LLM Agents](https://mlsanity.com/paper/2410.21286/) (2024)
- [Deep Learning for Cross-Domain Data Fusion in Urban Computing: Taxonomy, Advances, and Outlook](https://mlsanity.com/paper/2402.19348/) (2024)
- [LLM-based Human Simulations Have Not Yet Been Reliable](https://mlsanity.com/paper/2501.08579/) (2025)
- [UrbanWorld: An Urban World Model for 3D City Generation](https://mlsanity.com/paper/2407.11965/) (2024)
- [Foundation Models for Spatio-Temporal Data Science: A Tutorial and Survey](https://mlsanity.com/paper/2503.13502/) (2025)
- [Towards Urban General Intelligence: A Review and Outlook of Urban Foundation Models](https://mlsanity.com/paper/2402.01749/) (2024)
- [From Text to Transformation: A Comprehensive Review of Large Language Models' Versatility](https://mlsanity.com/paper/2402.16142/) (2024)
- [BIGCity: A Universal Spatiotemporal Model for Unified Trajectory and Traffic State Data Analysis](https://mlsanity.com/paper/2412.00953/) (2024)
- [UrbanLLaVA: A Multi-modal Large Language Model for Urban Intelligence with Spatial Reasoning and Understanding](https://mlsanity.com/paper/2506.23219/) (2025)
- [A Survey on Agent Workflow -- Status and Future](https://mlsanity.com/paper/2508.01186/) (2025)
- [YuLan-OneSim: Towards the Next Generation of Social Simulator with Large Language Models](https://mlsanity.com/paper/2505.07581/) (2025)
- [SmartAgent: Chain-of-User-Thought for Embodied Personalized Agent in Cyber World](https://mlsanity.com/paper/2412.07472/) (2024)
- [GATSim: Urban Mobility Simulation with Generative Agents](https://mlsanity.com/paper/2506.23306/) (2025)
- [Autonomous Workflow for Multimodal Fine-Grained Training Assistants Towards Mixed Reality](https://mlsanity.com/paper/2405.13034/) (2024)
- [Urban Region Pre-training and Prompting: A Graph-based Approach](https://mlsanity.com/paper/2408.05920/) (2024)
- [Modeling Earth-Scale Human-Like Societies with One Billion Agents](https://mlsanity.com/paper/2506.12078/) (2025)
- [UniFlow: A Foundation Model for Unified Urban Spatio-Temporal Flow Prediction](https://mlsanity.com/paper/2411.12972/) (2024)
- [UrbanPlanBench: A Comprehensive Urban Planning Benchmark for Evaluating Large Language Models](https://mlsanity.com/paper/2504.21027/) (2025)
- [LatticeWorld: A Multimodal Large Language Model-Empowered Framework for Interactive Complex World Generation](https://mlsanity.com/paper/2509.05263/) (2025)
- [Large Language Model Powered Intelligent Urban Agents: Concepts, Capabilities, and Applications](https://mlsanity.com/paper/2507.00914/) (2025)
- [Carbon and Silicon, Coexist or Compete? A Survey on Human-AI Interactions in Agent-based Modeling and Simulation](https://mlsanity.com/paper/2502.18145/) (2025)
- [Enhancing Traffic Prediction with Textual Data Using Large Language Models](https://mlsanity.com/paper/2405.06719/) (2024)

## Abstract (arXiv)

Urban environments, characterized by their complex, multi-layered networks encompassing physical, social, economic, and environmental dimensions, face significant challenges in the face of rapid urbanization. These challenges, ranging from traffic congestion and pollution to social inequality, call for advanced technological interventions. Recent developments in big data, artificial intelligence, urban computing, and digital twins have laid the groundwork for sophisticated city modeling and simulation. However, a gap persists between these technological capabilities and their practical implementation in addressing urban challenges in an systemic-intelligent way. This paper proposes Urban Generative Intelligence (UGI), a novel foundational platform integrating Large Language Models (LLMs) into urban systems to foster a new paradigm of urban intelligence. UGI leverages CityGPT, a foundation model trained on city-specific multi-source data, to create embodied agents for various urban tasks. These agents, operating within a textual urban environment emulated by city simulator and urban knowledge graph, interact through a natural language interface, offering an open platform for diverse intelligent and embodied agent development. This platform not only addresses specific urban issues but also simulates complex urban systems, providing a multidisciplinary approach to understand and manage urban complexity. This work signifies a transformative step in city science and urban intelligence, harnessing the power of LLMs to unravel and address the intricate dynamics of urban systems. The code repository with demonstrations will soon be released here https://github.com/tsinghua-fib-lab/UGI.