# Making Large Language Models Better Knowledge Miners for Online Marketing with Progressive Prompting Augmentation

arXiv: https://arxiv.org/abs/2312.05276 · HTML: https://mlsanity.com/paper/2312.05276/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 49 references that are in the map.
- First posted on arXiv 8 Dec 2023.

## Builds on (most cited first, 27 of 27)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [ConceptNet 5.5: An Open Multilingual Graph of General Knowledge](https://mlsanity.com/paper/1612.03975/) (2016)
- [GLM-130B: An Open Bilingual Pre-trained Model](https://mlsanity.com/paper/2210.02414/) (2022)
- [REPLUG: Retrieval-Augmented Black-Box Language Models](https://mlsanity.com/paper/2301.12652/) (2023)
- [Baichuan 2: Open Large-scale Language Models](https://mlsanity.com/paper/2309.10305/) (2023)
- [How Language Model Hallucinations Can Snowball](https://mlsanity.com/paper/2305.13534/) (2023)
- [Head-to-Tail: How Knowledgeable are Large Language Models (LLMs)? A.K.A. Will LLMs Replace Knowledge Graphs?](https://mlsanity.com/paper/2308.10168/) (2023)
- [Ask Me Anything: A simple strategy for prompting language models](https://mlsanity.com/paper/2210.02441/) (2022)
- [A Survey on Recent Advances in Named Entity Recognition from Deep Learning models](https://mlsanity.com/paper/1910.11470/) (2019)
- [Language Models are Open Knowledge Graphs](https://mlsanity.com/paper/2010.11967/) (2020)
- [Crawling the Internal Knowledge-Base of Language Models](https://mlsanity.com/paper/2301.12810/) (2023)
- [A Survey on Semantic Parsing](https://mlsanity.com/paper/1812.00978/) (2018)
- [TransOMCS: From Linguistic Graphs to Commonsense Knowledge](https://mlsanity.com/paper/2005.00206/) (2020)
- [Commonsense Properties from Query Logs and Question Answering Forums](https://mlsanity.com/paper/1905.10989/) (2019)
- [DISCOS: Bridging the Gap between Discourse Knowledge and Commonsense Knowledge](https://mlsanity.com/paper/2101.00154/) (2021)
- [Explicit Feature Interaction-aware Uplift Network for Online Marketing](https://mlsanity.com/paper/2306.00315/) (2023)
- [Optimizing Bi-Encoder for Named Entity Recognition via Contrastive Learning](https://mlsanity.com/paper/2208.14565/) (2022)
- [Soft Gazetteers for Low-Resource Named Entity Recognition](https://mlsanity.com/paper/2005.01866/) (2020)
- [Refined Commonsense Knowledge from Large-Scale Web Contents](https://mlsanity.com/paper/2112.04596/) (2021)
- [Learning to Expand Audience via Meta Hybrid Experts and Critics for Recommendation and Advertising](https://mlsanity.com/paper/2105.14688/) (2021)
- [Zero-shot Approach to Overcome Perturbation Sensitivity of Prompts](https://mlsanity.com/paper/2305.15689/) (2023)
- [Sources of Transfer in Multilingual Named Entity Recognition](https://mlsanity.com/paper/2005.00847/) (2020)
- [Adversarial Learning for Incentive Optimization in Mobile Payment Marketing](https://mlsanity.com/paper/2112.15434/) (2021)
- [Who Would be Interested in Services? An Entity Graph Learning System for User Targeting](https://mlsanity.com/paper/2305.18780/) (2023)
- [Exploring 360-Degree View of Customers for Lookalike Modeling](https://mlsanity.com/paper/2304.09105/) (2023)

## Built on by (most cited first, 5 of 5)

- [Large Language Models in Cybersecurity: State-of-the-Art](https://mlsanity.com/paper/2402.00891/) (2024)
- [Automated Construction of Theme-specific Knowledge Graphs](https://mlsanity.com/paper/2404.19146/) (2024)
- [A Survey on Retrieval And Structuring Augmented Generation with Large Language Models](https://mlsanity.com/paper/2509.10697/) (2025)
- [Robust Uplift Modeling with Large-Scale Contexts for Real-time Marketing](https://mlsanity.com/paper/2502.15697/) (2025)
- [Construction of Knowledge Graph based on Language Model](https://mlsanity.com/paper/2604.19137/) (2026)

## Abstract (arXiv)

Nowadays, the rapid development of mobile economy has promoted the flourishing of online marketing campaigns, whose success greatly hinges on the efficient matching between user preferences and desired marketing campaigns where a well-established Marketing-oriented Knowledge Graph (dubbed as MoKG) could serve as the critical "bridge" for preference propagation. In this paper, we seek to carefully prompt a Large Language Model (LLM) with domain-level knowledge as a better marketing-oriented knowledge miner for marketing-oriented knowledge graph construction, which is however non-trivial, suffering from several inevitable issues in real-world marketing scenarios, i.e., uncontrollable relation generation of LLMs,insufficient prompting ability of a single prompt, the unaffordable deployment cost of LLMs. To this end, we propose PAIR, a novel Progressive prompting Augmented mIning fRamework for harvesting marketing-oriented knowledge graph with LLMs. In particular, we reduce the pure relation generation to an LLM based adaptive relation filtering process through the knowledge-empowered prompting technique. Next, we steer LLMs for entity expansion with progressive prompting augmentation,followed by a reliable aggregation with comprehensive consideration of both self-consistency and semantic relatedness. In terms of online serving, we specialize in a small and white-box PAIR (i.e.,LightPAIR),which is fine-tuned with a high-quality corpus provided by a strong teacher-LLM. Extensive experiments and practical applications in audience targeting verify the effectiveness of the proposed (Light)PAIR.