Journals
  Publication Years
  Keywords
Search within results Open Search
Please wait a minute...
For Selected: Toggle Thumbnails
Twice attention mechanism distantly supervised relation extraction based on BERT
Quan YUAN, Changping CHEN, Ze CHEN, Linfeng ZHAN
Journal of Computer Applications    2024, 44 (4): 1080-1085.   DOI: 10.11772/j.issn.1001-9081.2023040490
Abstract330)   HTML10)    PDF (737KB)(487)       Save

Aiming at the problem of incomplete semantic information of word vectors and the problem of word polysemy faced by text feature extraction, a BERT (Bidirectional Encoder Representation from Transformer) word vector-based Twice Attention mechanism weighting algorithm for Relation Extraction (TARE) was proposed. Firstly, in the word embedding stage, the self-attention dynamic encoding algorithm was used to capture the semantic information before and after the text for the current word vector by constructing QK and V matrices. Then, after the model output the sentence-level feature vector, the locator was used to extract the corresponding parameters of the fully connected layer to construct the relation attention matrix. Finally, the sentence level attention mechanism algorithm was used to add different attention scores to sentence-level feature vectors to improve the noise immunity of sentence-level features. The experimental results show that compared with Contrastive Instance Learning (CIL) algorithm for relation extraction, the F1 value is increased by 4.0 percentage points and the average value of Precision@100, Precision@200, and Precision@300 (P@M) is increased by 11.3 percentage points on the NYT-10m dataset. Compared with the Piecewise Convolutional Neural Network algorithm based on ATTention mechanism (PCNN-ATT), the AUC (Area Under precision-recall Curve) value is increased by 4.8 percentage points and the P@M value is increased by 2.1 percentage points on the NYT-10d dataset. In various mainstream Distantly Supervised for Relation Extraction (DSRE) tasks, TARE effectively improves the model’s ability to learn data features.

Table and Figures | Reference | Related Articles | Metrics