Abstract: Because user prompts often lack professionalism in specific fields and the use of terminology, it is difficult for LLMs to accurately understand the intentions and generate information that meets the requirements of the field. Based on this, an Automated Preference Alignment Dual-Stage Prompt Tuning (APADPT) method has been pro-posed to solve the preference alignment problem faced by Large Language Models (LLMs) when applied in vertical fields. APADPT achieves the refinement adjustment of input prompts by constructing a supervised fine-tuning da-taset containing human preferences and using LLMs for semantic analysis and evaluation of pairwise replies. After two-stage training, the model not only masters the prompt optimization rules in the general field but also conducts specialized adjustments based on the characteristics of the vertical field. In the experiments in the medical field, APADPT significantly improved the preference alignment consistency of API-based LLMs and open-source LLMs, with the winning rate increasing by 9.5% to 20.5%. In addition, this method shows good robustness and generaliza-tion ability on open-source models with different parameter scales, providing a new optimization strategy for the application of LLMs in vertical specialized fields, contributing to achieving higher performance standards while maintaining the generalization and adaptability of the model.

Key words: large language model, vertical domain optimization, preference alignment, prompt optimization

摘要： 由于用户提示常常缺乏特定领域的专业性和术语使用，导致LLMs难以准确理解意图和生成符合领域要求的信息。基于此提出了一种自动化偏好对齐的双阶段提示调优方法（Automated Preference Alignment Dual-Stage Prompt Tuning, APADPT），以解决大型语言模型（LLMs）在垂直领域应用时面临的偏好对齐问题。APADPT通过构建包含人类偏好的监督微调数据集，并利用LLMs进行成对回复的语义分析和评估，实现对输入提示的精细化调整。经过两阶段训练，模型不仅掌握了通用领域的提示优化规律，还针对垂直领域特性进行了专业化调整。在医疗领域的实验中，APADPT显著提升了基于API的LLMs与开源LLMs的偏好对齐一致性，平均胜率提高9.5%至20.5%。此外，该方法在不同参数规模的开源模型上展现了良好的鲁棒性和泛化能力，为LLMs在垂直专业化领域中的应用提供了新的优化策略，有助于实现更高的性能标准，同时保持模型的泛化性和适应性。

关键词: 大语言模型, 垂直领域优化, 偏好对齐, 提示优化