关键词: 关系抽取, 分段卷积神经网络, 知识蒸馏, 知识图谱, 学科知识, 神经网络

Abstract:

Relational extraction is an important means of sorting out discipline knowledge as well as an important step in the construction of educational knowledge graph. In the current research， most of the pre-trained language models based on the Transformer architecture， such as the Bidirectional Encoder Representations from Transformers （BERT）， suffer from large number of parameters and excessive complexity， which make them difficult to be deployed on end devices and limite their applications in real educational scenarios. In addition， most traditional lightweight relation extraction models do not model the data through text structure， which are easy to ignore the structural information between entities， and the generated word embedding vectors are difficult to capture the contextual features of the text， have poor ability to solve the problem of multiple meanings of words， and are difficult to fit the unstructured nature of discipline knowledge texts and the high proportion of proper nouns， which is not conducive to high-quality relation extraction. In order to solve the above problems， a relation extraction method between discipline knowledge entities based on improved Piecewise Convolutional Neural Network （PCNN） and Knowledge Distillation （KD） was proposed. Firstly， BERT was used to generate high-quality domain text word vectors to improve the input layer of the PCNN model， so as to effectively capture the text context features and solve the problem of multiple meanings of words to a certain extent. Then， convolution and piecewise max pooling operations were utilized to deeply mine inter-entity structural information， constructing the BERT-PCNN model， and achieving high-quality relation extraction. Lastly， by taking into account the demands for efficient and lightweight models in educational scenarios， the knowledge of the output layer and middle layer of the BERT-PCNN model was distilled for guiding the PCNN model to complete the construction of the KD-PCNN model. The experimental results show that， the weighted-average F1 of the BERT-PCNN model reaches 94%， which is improved by 1 and 2 percentage points compared with the R-BERT and EC_BERT models； the weighted-average F1 of the KD-PCNN model reaches 92%， which is the same as the EC_BERT model， and the parameter quantity of the KD-PCNN model decreased by 3 orders of magnitude compared with the BERT-PCNN and KD-RB-l models. It can be seen that the proposed method can achieve a better trade-off between the performance evaluation index and the network parameter quantity， which is conducive to the improvement of the automated construction level of educational knowledge graph and the development and deployment of new educational applications.

Key words: relation extraction, Piecewise Convolution Neural Network (PCNN), knowledge distillation, knowledge graph, discipline knowledge, neural network