# Customized Generative AI Agent for Transportation Engineering Practice: A Development and Continued Pre-training Guideline

arXiv: https://arxiv.org/abs/2606.29014 · HTML: https://mlsanity.com/paper/2606.29014/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 22 of its 52 references that are in the map.
- First posted on arXiv 27 Jun 2026.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 22 of 22)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://mlsanity.com/paper/2404.14219/) (2024)
- [A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendations](https://mlsanity.com/paper/2407.04069/) (2024)
- [Internal Consistency and Self-Feedback in Large Language Models: A Survey](https://mlsanity.com/paper/2407.14507/) (2024)
- [TrafficSafetyGPT: Tuning a Pre-trained Large Language Model to a Domain-Specific Expert in Transportation Safety](https://mlsanity.com/paper/2307.15311/) (2023)
- [TSpec-LLM: An Open-source Dataset for LLM Understanding of 3GPP Specifications](https://mlsanity.com/paper/2406.01768/) (2024)
- [Can Large Language Models Serve as Data Analysts? A Multi-Agent Assisted Approach for Qualitative Data Analysis](https://mlsanity.com/paper/2402.01386/) (2024)
- [PolicyGPT: Automated Analysis of Privacy Policies with Large Language Models](https://mlsanity.com/paper/2309.10238/) (2023)
- [Using Large Language Models to Understand Telecom Standards](https://mlsanity.com/paper/2404.02929/) (2024)
- [Digital Twin-Assisted Data-Driven Optimization for Reliable Edge Caching in Wireless Networks](https://mlsanity.com/paper/2407.00286/) (2024)
- [Leveraging Fine-Tuned Retrieval-Augmented Generation with Long-Context Support: For 3GPP Standards](https://mlsanity.com/paper/2408.11775/) (2024)
- [Digital Network Twins for Next-generation Wireless: Creation, Optimization, and Challenges](https://mlsanity.com/paper/2410.18002/) (2024)
- [INSIGHT: Enhancing Autonomous Driving Safety through Vision-Language Models on Context-Aware Hazard Detection and Edge Case Evaluation](https://mlsanity.com/paper/2502.00262/) (2025)
- [Using Collision Momentum in Deep Reinforcement Learning Based Adversarial Pedestrian Modeling](https://mlsanity.com/paper/2306.07525/) (2023)
- [Deep Reinforcement Learning for Advanced Longitudinal Control and Collision Avoidance in High-Risk Driving Scenarios](https://mlsanity.com/paper/2404.19087/) (2024)
- [Large Language Model in Financial Regulatory Interpretation](https://mlsanity.com/paper/2405.06808/) (2024)
- [Enhancing Travel Choice Modeling with Large Language Models: A Prompt-Learning Approach](https://mlsanity.com/paper/2406.13558/) (2024)
- [Synergizing AI and Digital Twins for Next-Generation Network Optimization, Forecasting, and Security](https://mlsanity.com/paper/2503.06302/) (2025)
- [Risks and NLP Design: A Case Study on Procedural Document QA](https://mlsanity.com/paper/2408.11860/) (2024)
- [Multiclass Classification of Policy Documents with Large Language Models](https://mlsanity.com/paper/2310.08167/) (2023)

## Abstract (arXiv)

Recent advancements in generative artificial intelligence (AI) and large language models (LLMs) have shown significant promise in automating complex reasoning, summarization, and question-answering tasks. However, the effectiveness of general-purpose LLMs in specialized engineering domains remains limited due to insufficient exposure to technical standards, engineering terminology, and domain-specific semantics. This study proposes a systematic approach to developing a customized generative AI agent for transportation engineering applications. A curated corpus of U.S. transportation manuals, design guidelines, and regulatory documents is used to conduct continued pretraining of six state-of-the-art LLMs through a unified low-rank adaptation (LoRA) framework. The training process is monitored to ensure convergence and model stability. Performance is evaluated using standard natural language processing metrics, including BLEU-4 and ROUGE, with Qwen2.5-7B and LLaMA-3.1-8B demonstrating the highest domain alignment and response quality. Results validate the effectiveness of LoRA-based adaptation in improving LLM performance on technical content interpretation and context-specific reasoning. This work contributes a reproducible development framework for constructing domain-specialized generative AI agents, supporting broader deployment in transportation research, design, planning, and policy analysis.