关键词: 大语言模型, 垂直领域优化, 偏好对齐, 提示优化

Abstract:

Because user prompts often lack professionalism in specific fields and the use of terminology， it is difficult for Large Language Models （LLM） to understand the intentions accurately and generate information that meets requirements of the field. Therefore， an Automated Preference Alignment Dual-Stage Prompt Tuning （APADPT） method was proposed to solve the preference alignment problem faced by LLM when applied in vertical fields. In APADPT， the refinement adjustments of input prompts were realized by constructing a supervised fine-tuning dataset containing human preferences and using LLM for semantic analysis and evaluation of pairwise replies. After dual-stage training， the prompt optimization rules in the general field were mastered by the model， and specialized adjustments based on characteristics of the vertical fields were performed by the model. Experimental results in the medical field show that APADPT improves the preference alignment consistency of API-based LLM and open-source LLM significantly， with the average winning rate increased by 9.5% to 20.5% under the condition of the same model parameter count. In addition， this method shows good robustness and generalization ability on all the open-source models with different parameter scales， providing a new optimization strategy for the application of LLM in vertical specialized fields， and contributing to improving model performance while maintaining generalization and adaptability of the model.

Key words: Large Language Model (LLM), vertical field optimization, preference alignment, prompt optimization