# Kuaipedia: a Large-scale Multi-modal Short-video Encyclopedia

arXiv: https://arxiv.org/abs/2211.00732 · HTML: https://mlsanity.com/paper/2211.00732/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 56 references that are in the map.
- First posted on arXiv 28 Oct 2022.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Neural Architectures for Named Entity Recognition](https://mlsanity.com/paper/1603.01360/) (2016)
- [Bidirectional LSTM-CRF Models for Sequence Tagging](https://mlsanity.com/paper/1508.01991/) (2015)
- [Pre-trained Models for Natural Language Processing: A Survey](https://mlsanity.com/paper/2003.08271/) (2020)
- [Autoregressive Entity Retrieval](https://mlsanity.com/paper/2010.00904/) (2020)
- [MERLOT: Multimodal Neural Script Knowledge Models](https://mlsanity.com/paper/2106.02636/) (2021)
- [Named Entity Recognition with Bidirectional LSTM-CNNs](https://mlsanity.com/paper/1511.08308/) (2015)
- [MMKG: Multi-Modal Knowledge Graphs](https://mlsanity.com/paper/1903.05485/) (2019)
- [Joint Extraction of Entities and Relations Based on a Novel Tagging Scheme](https://mlsanity.com/paper/1706.05075/) (2017)
- [End-to-End Neural Entity Linking](https://mlsanity.com/paper/1808.07699/) (2018)
- [Multi-Modal Knowledge Graph Construction and Application: A Survey](https://mlsanity.com/paper/2202.05786/) (2022)
- [Improving Entity Linking by Modeling Latent Relations between Mentions](https://mlsanity.com/paper/1804.10637/) (2018)
- [Investigating Entity Knowledge in BERT with Simple Neural End-To-End Entity Linking](https://mlsanity.com/paper/2003.05473/) (2020)
- [Multimodal Named Entity Recognition for Short Social Media Posts](https://mlsanity.com/paper/1802.07862/) (2018)
- [Fast and Accurate Entity Recognition with Iterated Dilated Convolutions](https://mlsanity.com/paper/1702.02098/) (2017)
- [MuKEA: Multimodal Knowledge Extraction and Accumulation for Knowledge-based Visual Question Answering](https://mlsanity.com/paper/2203.09138/) (2022)
- [VisualSem: A High-quality Knowledge Graph for Vision and Language](https://mlsanity.com/paper/2008.09150/) (2020)
- [Multimodal Entity Linking for Tweets](https://mlsanity.com/paper/2104.03236/) (2021)
- [NSP-BERT: A Prompt-based Few-Shot Learner Through an Original Pre-training Task--Next Sentence Prediction](https://mlsanity.com/paper/2109.03564/) (2021)
- [Learning Dynamic Context Augmentation for Global Entity Linking](https://mlsanity.com/paper/1909.02117/) (2019)
- [Boosting Entity Linking Performance by Leveraging Unlabeled Documents](https://mlsanity.com/paper/1906.01250/) (2019)
- [Toward Mention Detection Robustness with Recurrent Neural Networks](https://mlsanity.com/paper/1602.07749/) (2016)

## Built on by (most cited first, 8 of 8)

- [Knowledge Graphs Meet Multi-Modal Learning: A Comprehensive Survey](https://mlsanity.com/paper/2402.05391/) (2024)
- [NativE: Multi-modal Knowledge Graph Completion in the Wild](https://mlsanity.com/paper/2406.17605/) (2024)
- [KwaiAgents: Generalized Information-seeking Agent System with Large Language Models](https://mlsanity.com/paper/2312.04889/) (2023)
- [Multiple Heads are Better than One: Mixture of Modality Knowledge Experts for Entity Representation Learning](https://mlsanity.com/paper/2405.16869/) (2024)
- [CogGPT: Unleashing the Power of Cognitive Dynamics on Large Language Models](https://mlsanity.com/paper/2401.08438/) (2024)
- [Abstractive Visual Understanding of Multi-modal Structured Knowledge: A New Perspective for MLLM Evaluation](https://mlsanity.com/paper/2506.01293/) (2025)
- [Complementarity-driven Representation Learning for Multi-modal Knowledge Graph Completion](https://mlsanity.com/paper/2507.20620/) (2025)
- [Evidence-Grounded Multimodal Knowledge Graph Construction for Multi-Lecture Educational Reasoning](https://mlsanity.com/paper/2608.03161/) (2026)

## Abstract (arXiv)

Online encyclopedias, such as Wikipedia, have been well-developed and researched in the last two decades. One can find any attributes or other information of a wiki item on a wiki page edited by a community of volunteers. However, the traditional text, images and tables can hardly express some aspects of an wiki item. For example, when we talk about ``Shiba Inu'', one may care more about ``How to feed it'' or ``How to train it not to protect its food''. Currently, short-video platforms have become a hallmark in the online world. Whether you're on TikTok, Instagram, Kuaishou, or YouTube Shorts, short-video apps have changed how we consume and create content today. Except for producing short videos for entertainment, we can find more and more authors sharing insightful knowledge widely across all walks of life. These short videos, which we call knowledge videos, can easily express any aspects (e.g. hair or how-to-feed) consumers want to know about an item (e.g. Shiba Inu), and they can be systematically analyzed and organized like an online encyclopedia. In this paper, we propose Kuaipedia, a large-scale multi-modal encyclopedia consisting of items, aspects, and short videos lined to them, which was extracted from billions of videos of Kuaishou (Kwai), a well-known short-video platform in China. We first collected items from multiple sources and mined user-centered aspects from millions of users' queries to build an item-aspect tree. Then we propose a new task called ``multi-modal item-aspect linking'' as an expansion of ``entity linking'' to link short videos into item-aspect pairs and build the whole short-video encyclopedia. Intrinsic evaluations show that our encyclopedia is of large scale and highly accurate. We also conduct sufficient extrinsic experiments to show how Kuaipedia can help fundamental applications such as entity typing and entity linking.