Abstract:

A government affairs hotline Question Answering （QA） system based on knowledge-enhanced Large Language Model （LLM） architecture， namely ChatGovt， was proposed to address the issues in current systems， such as low manual response efficiency， as well as inaccurate query screening mechanisms and insufficient intention difference recognition in traditional Retrieval-Augmented Generation （RAG） systems. Firstly， to improve response efficiency， a system architecture integrating intelligent question diverting and structured feedback was designed， thereby enabling classified processing of problems of consultation， complaint/suggestion， and other types with intent recognition. Then， to improve the system’s knowledge retrieval quality， a multi-stage semantic-enhanced retrieval method was proposed， including three stages： historical dialogue summary retrieval， semantic re-ranking， and self-reflective decision-making. Finally， cross-domain knowledge was supplemented through online queries， so as to form a service closed-loop for government consultation. Experimental results show that in terms of retrieval quality， compared with the traditional RAG system， ChatGovt has the query-knowledge relevance， real answer-knowledge relevance， and knowledge support improved by 15.0%， 7.4%， and 24.6% respectively； in terms of overall system performance， ChatGovt has the answer recall increased by 55.4% compared with the fine-tuned GLM （General Language Model）4-9b-chat model， and has the manual evaluation improved by 27.3% compared with the commercial system “Doubao”. It can be seen that this system provides a reference-worthy architecture and methodology for the technical optimization of government affairs hotline QA systems， can improve the response efficiency and service accuracy of government affairs hotlines effectively， and promotes the intelligent transformation of government services.

Key words: government affairs Question Answering (QA) system, Retrieval-Augmented Generation (RAG), Large Language Model (LLM), semantic retrieval, knowledge enhancement

摘要：

针对当前政务热线问答（QA）系统中人工回复效率低以及传统检索增强生成（RAG）系统存在查询甄别机制不精准和意图差异识别不足等问题，提出一种基于知识增强大语言模型（LLM）架构的政务热线QA系统ChatGovt。首先，为了提高回复效率，设计整合智能问题分流和结构化反馈的系统架构，通过意图识别实现咨询类和投诉建议类等问题的分类处理；其次，为了提高系统检索知识的质量，提出多阶段语义增强的检索方法，包括历史对话总结检索、语义重排序和自我反思决策这3个阶段；最后，通过联网查询补充跨域知识，以形成政务咨询的服务闭环。实验结果表明，在检索质量上，相较于传统RAG系统，ChatGovt的查询-知识相关性、真实答案-知识相关性和知识支持度分别提升了15.0%、7.4%和24.6%；在系统整体性能上，ChatGovt的答案召回率相较于微调的GLM（General Language Model）4-9b-chat提升了55.4%，人工评价相较于商用系统“豆包”提升了27.3%。可见，该系统为政务热线QA系统的技术优化提供了可借鉴的架构与方法，能有效提升政务热线的响应效率与服务精准度，并推动政务服务的智能化转型。

关键词: 政务问答系统, 检索增强生成, 大语言模型, 语义检索, 知识增强