Abstract:

To address the issues of ignoring constraint information and the curse of dimensionality in long-path reasoning in Knowledge Graph Question Answering （KGQA）， a Knowledge Graph constrained Question Answering based on Hierarchical Reinforcement Learning （HRL） （KGQA-HRL） model was proposed. Firstly， the concept of HRL was integrated deeply， triples in the knowledge graph were decomposed， and a high-level policy as well as a low-level policy was designed， so as to mitigate the curse of dimensionality risk in reasoning paths. Secondly， to improve the accuracy of path selection， an attention-based action selection strategy and an entity selection strategy incorporating constraint information were introduced， thereby narrowing the search space for reasoning effectively. Thirdly， a question update phase was embedded between the action selection and entity selection strategies， thereby enabling secondary update of the question at each hop. Finally， in the entity selection strategy， a constraint set was constructed and constraint scores were calculated， so as to incorporate constraint information from the question， thereby enhancing the accuracy of entity selection. Experimental results on four KGQA benchmark datasets to evaluate the performance of KGQA-HRL model demonstrate that KGQA-HRL model achieves the optimal accuracy on all datasets， with an average improvement of 2.9% over the previous best model reinforcement learning based COnstrained PAth Reasoning （COPAR）. At the same time， KGQA-HRL model has outstanding performance in complex three-hop query tasks （3.6% improvement on the PQ （PathQuestion） dataset and 2.5% improvement on MetaQA dataset）， validating good reasoning capability of KGQA-HRL model.

Key words: Knowledge Graph Question Answering (KGQA), Hierarchical Reinforcement Learning (HRL), complex constrained problem, attention mechanism, curse of dimensionality

摘要：

针对知识图谱问答（KGQA）中忽略约束信息和长路径推理的维数灾难问题，提出一种基于分层强化学习（HRL）的知识图谱约束问答（KGQA-HRL）模型。首先，深度融合HRL理念，拆解知识图谱中的三元组，并设计出上层策略和底层策略，从而化解推理路径的维数灾难隐患；其次，为了提高路径选择的准确性，提出基于注意力机制的动作选择策略和融合约束信息的实体选择策略，从而有效压缩推理的搜索范围；再次，在动作选择与实体选择策略之间嵌入问题更新环节，从而使每跳问题进行二次更新；最后，在实体选择策略中构建约束集并计算约束得分，以考虑问题中的约束信息，从而提高实体选择的准确性。基于4个KGQA基准数据集，对KGQA-HRL模型的性能进行实验的结果显示：KGQA-HRL模型在所有数据集上均达到最优准确率，较之前最佳模型约束路径推理（COPAR）提升了2.9%，且在复杂的3跳查询任务中表现突出（PQ （PathQuestion）数据集上提升了3.6%，MetaQA数据集上提升了2.5%），验证了KGQA-HRL模型优异的推理能力。

关键词: 知识图谱问答, 分层强化学习, 复杂约束问题, 注意力机制, 维度灾难