Abstract: Relational extraction as an important means of sorting out discipline knowledge as well as an important step in the construction of educational knowledge graph, in the current research, most of the pre-trained language models based on the Transformer architecture, such as the Bidirectional Encoder Representations from Transformers (BERT), suffered from large number of parameters and excessive complexity, which made it difficult to be deployed on end devices and limited their application in real educational scenarios. In addition, most traditional lightweight relational extraction models did not model the data through text structure, which was easy to ignore the structural information between entities, and the generated word embedding vectors were difficult to capture the contextual features of the text, have poor ability to solve the problem of multiple meanings of words, and were difficult to fit the unstructured nature of discipline knowledge texts and the high proportion of proper nouns, which was not conducive to high-quality relational extraction. In order to solve the above problems, a relationship extraction method between discipline knowledge entities based on improved Piecewise Convolutional Neural Network (PCNN) and Knowledge Distillation (KD) was proposed. Firstly, BERT was used to generate high-quality domain text word vectors to improve the input layer of the PCNN model, so as to effectively capture the text context features and solve the problem of multiple meanings of words to a certain extent. Then, convolution and piecewise maximal pooling operations were utilized to deeply mine inter-entity structural information, completed the construction of the BERT-PCNN model, and achieved high-quality relationship extraction. Lastly, the BERT-PCNN model was constructed by taking into account the the demand for efficient and lightweight models in educational scenarios, distilled the knowledge of the output layer and middle layer of the BERT-PCNN model for guiding the PCNN model, and completed the construction of the KD-PCNN model. The experimental results show that the Weighted-average F1 of the BERT-PCNN model reaches 94%, which is improved by 1 percentage point and 2 percentage points compared with the R-BERT and EC-BERT models; the Weighted-average F1 of the KD-PCNN model reaches 92%, which is same as the EC-BERT model, and the parameter quantity decreased by 3 orders of magnitude compared with the BERT-PCNN and KD-RB-l models. It can be seen that the proposed method can achieve a better trade-off between the performance evaluation index and the network parameter quantity, which was conducive to the improvement of the degree of automated construction of educational knowledge graph and the development and deployment of new educational applications.

Key words: relation extraction, Piecewise Convolution Neural Network (PCNN), Bidirectional Encoder Representations from Transformers (BERT), knowledge distillation, knowledge graph, discipline knowledge, neural network

摘要： 关系抽取作为梳理学科知识的重要手段以及教育知识图谱构建的重要步骤，在当前研究中，如基于变换器的双向编码器表示技术(BERT)等以Transformer架构为基础的预训练语言模型多数存在参数量大、复杂程度过高的问题，导致其难以部署到终端设备上，在真实教育场景中的应用受到限制。此外，大多数传统的轻量级关系抽取模型并不是通过文本结构对数据进行建模，容易忽略实体间的结构信息，且生成的词嵌入向量难以捕捉文本的上下文特征、一词多义问题解决能力差，难以契合学科知识文本非结构化以及专有名词占比大的特点，不利于高质量的关系抽取。针对上述问题，提出了一种基于改进分段卷积神经网络(PCNN)和知识蒸馏(KD)的学科知识实体间关系抽取方法。首先，利用BERT生成高质量的领域文本词向量，对PCNN模型的输入层进行改进，从而有效捕捉文本上下文特征并在一定程度上解决一词多义问题；其次，利用卷积和分段最大池化操作深入挖掘实体间结构信息，完成BERT-PCNN模型构建，实现高质量的关系抽取；最后，考虑到教育场景对高效且轻量化模型的需求，蒸馏BERT-PCNN模型输出层和中间层知识，用于指导PCNN模型，完成KD-PCNN模型的构建。实验结果表明，BERT-PCNN模型的Weighted-average F1达到94%，相较于R-BERT和EC-BERT模型分别提升了1个百分点和2个百分点；KD-PCNN模型的Weighted-average F1达到92%，持平于EC-BERT模型，参数量相较于BERT-PCNN、KD-RB-l模型下降了3个数量级。可见，所提方法能够在性能评价指标和网络参数量之间取得更好的权衡，有利于教育知识图谱自动化构建程度的提高和新型教育应用的研发与部署。

关键词: 关系抽取, 分段卷积神经网络, 基于变换器的双向编码器表示技术, 知识蒸馏, 知识图谱, 学科知识, 神经网络