Aiming at the multi?classification problem for Motor Imagery ElectroEncephaloGraphy (MI?EEG), Lightweight convolutional neural Network (L?Net) and Lightweight Hybrid Network (LH?Net) based on deep separable convolution were built on the basis of existing research. Experiments and analyses were carried out on the BCI competition IV-2a data set. It was shown that L?Net could fit the data faster than LH?Net, and the training time was shorter. However, LH?Net is more stable than L?Net and has better robustness in classification performance on the test set, the average accuracy and average Kappa coefficient of LH?Net were increased by 3.6% and 4.8%, respectively compared with L?Net. In order to further improve the classification performance of the model, a new method of adding Gaussian noise based on the time?frequency domain was adopted to apply Data Augmentation (DA) on the training samples, and simulation verification of the noise intensity was carried out, thus the optimal noise intensity ranges of the two models were inferred. With the DA method, the average accuracies of the two models were increased by at least 4% in the simulation results, the four classification effects were significantly improved.
The development of pre-trained language models has greatly promoted the progress of machine reading comprehension tasks. In order to make full use of shallow features of the pre-trained language model and further improve the accuracy of predictive answer of question answering model, a three-stage question answering model based on Bidirectional Encoder Representation from Transformers (BERT) was proposed. Firstly, the three stages of pre-answering, re-answering and answer-adjusting were designed based on BERT. Secondly, the inputs of embedding layer of BERT were treated as shallow features to pre-generate an answer in pre-answering stage. Then, the deep features fully encoded by BERT were used to re-generate another answer in re-answering stage. Finally, the final prediction result was generated by combining the previous two answers in answer-adjusting stage. Experimental results on English dataset Stanford Question Answering Dataset 2.0 (SQuAD2.0) and Chinese dataset Chinese Machine Reading Comprehension 2018 (CMRC2018) of span-extraction question answering task show that the Exact Match (EM) and F1 score (F1) of the proposed model are improved by the average of 1 to 3 percentage points compared with those of the similar baseline models, and the model has the extracted answer fragments more accurate. By combining shallow features of BERT with deep features, this three-stage model extends the abstract representation ability of BERT, and explores the application of shallow features of BERT in question answering models, and has the characteristics of simple structure, accurate prediction, and fast speed of training and inference.
Aiming at the issues in fault diagnosis of energy Internet such as long model training time, insufficient extraction of fault features, and low diagnostic accuracy with limited training sample size, a Hierarchical Clustering and Multi-Head attention based Convolutional neural network (HCMHC) model was proposed. In the model, the novel Hierarchical Clustering (HC) model was adopted to reduce data redundancy effectively, while Convolutional Neural Network (CNN) and multi-head attention were combined for more accurate and comprehensive fault feature extraction. Furthermore, a contrastive learning model was employed to enhance the complementarity among features with limited training sample size, thereby improving model generalization ability and diagnostic accuracy on new data. Experimental verification results on the New England test system with 39 buses and 10 generators demonstrate that the HCMHC model achieves accuracies of 99.8% and 99.5% on two different datasets respectively, which have improvements of 4.3 and 4.5 percentage points approximately and respectively compared to the Multiple-Input CNN (MI-CNN) model. Additionally, even with a training set/validation set ratio of 20/80, this model still has accuracies of 98.3% and 95.8% on two datasets respectively. The above proves the significant effectiveness and superiority of the proposed model in the field of fault diagnosis.