mlsanityData to 14 Oct 2026

DeepThink: Aligning Language Models with Domain-Specific User Intents

arXiv 2502.05497 ↗First posted 8 Feb 2025cs.CLcited by 2

On the map: LLM reasoning → Test-time compute and efficient thinking

also in 1 more
Abstract (arXiv)

Supervised fine-tuning with synthesized instructions has been a common practice for adapting LLMs to domain-specific QA tasks. However, the synthesized instructions deviate from real user questions and expected answers. This study proposes a novel framework called DeepThink to generate high-quality instructions. DeepThink first generates a few seed questions to mimic actual user questions, simulates conversations to uncover the hidden user needs, and refines the answer by conversational contexts and the retrieved documents for more comprehensive answers. Experiments demonstrate that DeepThink achieves an average performance improvement of 7.92% compared to a GPT-4-turbo+RAG-based assistant on the real user test set in the advertising domain across dimensions such as relevance, completeness, clarity, accuracy, and actionability.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 15 of its 36 references that are in the map. First posted on arXiv 8 Feb 2025.

Builds on

15 of 36 references.

References · 15

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023Mistral 7Bcited by 3,164
  3. 2023BloombergGPT: A Large Language Model for Financecited by 939
  4. 2023LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Datasetcited by 461
  5. 2023JudgeLM: Fine-tuned Large Language Models are Scalable Judgescited by 367
  6. 2024Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothingcited by 346
  7. 2024RAFT: Adapting Language Model to Domain Specific RAGcited by 262
  8. 2023Query Rewriting for Retrieval-Augmented Large Language Modelscited by 213
  9. 2024ChatQA: Surpassing GPT-4 on Conversational QA and RAGcited by 90
  10. 2023GPTEval: A Survey on Assessments of ChatGPT and GPT-4cited by 73
  11. 2024Synthetic Dialogue Dataset Generation using LLM Agentscited by 39
  12. 2024T-RAG: Lessons from the LLM Trenchescited by 23
3 more
  1. 2024CantTalkAboutThis: Aligning Language Models to Stay on Topic in Dialoguescited by 15
  2. 2024Instruct Once, Chat Consistently in Multiple Rounds: An Efficient Tuning Framework for Dialoguecited by 11
  3. 2023Self-QA: Unsupervised Knowledge Guided Language Model Alignmentcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Role-Augmented Intent-Driven Generative Search Engine Optimizationcited by 5
  2. 2026From Experience to Skill: Multi-Agent Generative Engine Optimization via Reusable Strategy Learningcited by 4

Similar, not cited

Close in wording, no citation either way

  1. 2025DeepQuestion: Systematic Generation of Real-World Challenges for Evaluating LLMs Performance
  2. 2025On Synthesizing Data for Context Attribution in Question Answering
  3. 2024Putting People in LLMs' Shoes: Generating Better Answers via Question Rewriter
  4. 2023ExpertPrompting: Instructing Large Language Models to be Distinguished Experts
  5. 2026TAMUSA-Chat: A Domain-Adapted Large Language Model Conversational System for Research and Responsible Deployment
  6. 2025Leveraging Reasoning Model Answers to Enhance Non-Reasoning Model Capability
  7. 2024Enhancing Q&A with Domain-Specific Fine-Tuning and Iterative Reasoning: A Comparative Study
  8. 2024Balancing Enhancement, Harmlessness, and General Capabilities: Enhancing Conversational LLMs with Direct RLHF