Abstract:

Focusing on problems that Large Language Models （LLMs） may generate hallucinations and are difficult to be fully applied to various fields of real life， especially medical field， as well as there is no high-quality LLM hallucination evaluation dataset and corresponding LLM hallucination degree evaluation， a method for identifying and optimizing LLM hallucinations in medical question answering field was proposed. Firstly， based on the publicly available dataset Huatuo， an LLM hallucination evaluation dataset in medical question answering field was constructed by combining GPT-4 generated question answers and manual annotation. Secondly， based on the constructed hallucination evaluation dataset， the concept of “hallucination rate” was defined. By designing prompts for the models to be tested answering “yes” or “no”， the degree of hallucination of each LLM was tested and quantified， and the “YES MAN” hallucination phenomenon of LLM was discovered. Thirdly， a low hallucination rate LLM， GPT-4， was used as LeaderAI to provide prior knowledge to assist LLMs with high hallucination rate in making judgments. Finally， to explore whether multiple different LLMs will make mistakes on the same problem， the concept of “hallucination collision” was defined， and based on probability statistical method， the hallucination collision situations of different LLMs in medical question answering field were revealed. Experimental results show that the introduction of LeaderAI can improve the performance of LLMs with high hallucination rate， so that LLMs can handle with the “YES MAN” hallucination phenomenon in medical question answering with low hallucination rate. Moreover， the current LLMs have a low probability of having hallucinations on a single question （collisions）.

Key words: Large Language Model (LLM), hallucination recognition, hallucination rate, hallucination collision, model optimization

摘要：

针对大语言模型（LLM）会产生幻觉，难以完全应用到现实生活各个领域（尤其是医疗领域），以及没有高质量的LLM幻觉评估数据集及相应的LLM幻觉程度评估的问题，提出在医疗问答领域中的LLM幻觉识别与优化方法。首先，根据公开数据集Huatuo，结合GPT-4生成问题答案和人工标注的形式构建医疗问答领域LLM幻觉评估数据集；其次，基于所构建的幻觉评估数据集，定义“幻觉率”的概念，通过设计prompt让待测模型回答“是”或“否”的方式测试和量化各个LLM的幻觉程度，并发现LLM的“YES MAN”幻觉现象；再次，采用低幻觉率的大模型GPT-4作为LeaderAI来提供先验知识辅助高幻觉率LLM进行判断；最后，为探究多个不同LLM是否会在同一个问题上犯错，定义“幻觉碰撞”的概念，并基于概率统计方法揭示不同LLM在医疗问答领域的幻觉碰撞情况。实验结果表明，引入LeaderAI的方法可以提升高幻觉率LLM的表现，使LLM能够以低幻觉率应对医疗问答领域的“YES MAN”幻觉现象，并且目前的LLM同时在一个问题上出现幻觉（发生碰撞）的概率较低。

关键词: 大语言模型, 幻觉识别, 幻觉率, 幻觉碰撞, 模型优化