Journal of Computer Applications ›› 2022, Vol. 42 ›› Issue (11): 3625-3632.DOI: 10.11772/j.issn.1001-9081.2021091701
Special Issue: 人工智能
• Artificial intelligence • Previous Articles Next Articles
Yu PENG, Yaolian SONG(), Jun YANG
Received:
2021-09-30
Revised:
2022-01-05
Accepted:
2022-01-28
Online:
2022-11-14
Published:
2022-11-10
Contact:
Yaolian SONG
About author:
PENG Yu, born in 1995, M. S. candidate. His research interests include brain information decoding, deep learning.通讯作者:
宋耀莲
作者简介:
彭禹(1995—),男,四川泸州人,硕士研究生,CCF会员,主要研究方向:脑信息解码、深度学习CLC Number:
Yu PENG, Yaolian SONG, Jun YANG. Motor imagery electroencephalography classification based on data augmentation[J]. Journal of Computer Applications, 2022, 42(11): 3625-3632.
彭禹, 宋耀莲, 杨俊. 基于数据增强的运动想象脑电分类[J]. 《计算机应用》唯一官方网站, 2022, 42(11): 3625-3632.
Add to citation manager EndNote|Ris|BibTeX
URL: https://www.joca.cn/EN/10.11772/j.issn.1001-9081.2021091701
受试者 | FBCSP | MSFBCNN[ | EEGNet[ | 3DCNN[ | L‑Net | LH‑Net | ||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Accuracy | Kappa | Accuracy | Kappa | Accuracy | Kappa | Accuracy | Kappa | Accuracy | Kappa | Accuracy | Kappa | |
均值 | 66.18 | 54.88 | 74.91 | 66.50 | 73.42 | 64.53 | 76.22 | 68.30 | 74.22 | 65.62 | 77.83 | 70.44 |
A01 | 73.44 | 64.57 | 81.60 | 75.41 | 83.33 | 77.73 | 81.22 | 74.96 | 81.14 | 74.86 | 79.36 | 72.48 |
A02 | 56.08 | 41.41 | 64.15 | 52.20 | 63.80 | 51.68 | 67.95 | 57.27 | 64.66 | 52.89 | 64.31 | 52.46 |
A03 | 80.42 | 73.88 | 86.98 | 82.60 | 88.76 | 84.98 | 84.09 | 78.78 | 84.98 | 79.96 | 91.57 | 88.76 |
A04 | 57.68 | 43.59 | 68.14 | 57.42 | 62.41 | 49.92 | 65.69 | 54.25 | 71.05 | 61.31 | 70.18 | 60.19 |
A05 | 57.38 | 43.13 | 71.27 | 61.58 | 58.72 | 44.97 | 70.58 | 60.78 | 71.37 | 61.87 | 74.63 | 66.18 |
A06 | 49.48 | 32.63 | 63.37 | 51.11 | 58.51 | 44.62 | 67.57 | 56.76 | 56.28 | 41.63 | 66.97 | 55.97 |
A07 | 81.25 | 74.97 | 90.54 | 87.36 | 84.81 | 79.72 | 85.87 | 81.16 | 83.03 | 77.42 | 85.56 | 80.78 |
A08 | 73.52 | 64.63 | 77.87 | 70.47 | 82.12 | 76.08 | 84.94 | 79.91 | 77.86 | 70.48 | 83.02 | 77.36 |
A09 | 66.36 | 55.12 | 70.31 | 60.35 | 78.30 | 71.06 | 78.10 | 70.80 | 77.65 | 70.20 | 84.84 | 79.78 |
Tab. 1 Comparison of classification performance of each method on test set
受试者 | FBCSP | MSFBCNN[ | EEGNet[ | 3DCNN[ | L‑Net | LH‑Net | ||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Accuracy | Kappa | Accuracy | Kappa | Accuracy | Kappa | Accuracy | Kappa | Accuracy | Kappa | Accuracy | Kappa | |
均值 | 66.18 | 54.88 | 74.91 | 66.50 | 73.42 | 64.53 | 76.22 | 68.30 | 74.22 | 65.62 | 77.83 | 70.44 |
A01 | 73.44 | 64.57 | 81.60 | 75.41 | 83.33 | 77.73 | 81.22 | 74.96 | 81.14 | 74.86 | 79.36 | 72.48 |
A02 | 56.08 | 41.41 | 64.15 | 52.20 | 63.80 | 51.68 | 67.95 | 57.27 | 64.66 | 52.89 | 64.31 | 52.46 |
A03 | 80.42 | 73.88 | 86.98 | 82.60 | 88.76 | 84.98 | 84.09 | 78.78 | 84.98 | 79.96 | 91.57 | 88.76 |
A04 | 57.68 | 43.59 | 68.14 | 57.42 | 62.41 | 49.92 | 65.69 | 54.25 | 71.05 | 61.31 | 70.18 | 60.19 |
A05 | 57.38 | 43.13 | 71.27 | 61.58 | 58.72 | 44.97 | 70.58 | 60.78 | 71.37 | 61.87 | 74.63 | 66.18 |
A06 | 49.48 | 32.63 | 63.37 | 51.11 | 58.51 | 44.62 | 67.57 | 56.76 | 56.28 | 41.63 | 66.97 | 55.97 |
A07 | 81.25 | 74.97 | 90.54 | 87.36 | 84.81 | 79.72 | 85.87 | 81.16 | 83.03 | 77.42 | 85.56 | 80.78 |
A08 | 73.52 | 64.63 | 77.87 | 70.47 | 82.12 | 76.08 | 84.94 | 79.91 | 77.86 | 70.48 | 83.02 | 77.36 |
A09 | 66.36 | 55.12 | 70.31 | 60.35 | 78.30 | 71.06 | 78.10 | 70.80 | 77.65 | 70.20 | 84.84 | 79.78 |
受试者 | L‑Net | L‑Net(NA) | L‑Net+(STFT+NA) | LH‑Net | LH‑Net(NA) | LH‑Net+(STFT+NA) | ||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Acc | Kappa | Acc | Kappa | Acc | Kappa | Acc | Kappa | Acc | Kappa | Acc | Kappa | |
均值 | 74.22 | 65.62 | 75.14 | 66.97 | 79.12 | 72.17 | 77.83 | 70.44 | 79.28 | 72.39 | 82.67 | 76.90 |
A01 | 81.14 | 74.86 | 81.85 | 75.81 | 82.56 | 76.73 | 79.36 | 72.48 | 80.07 | 73.43 | 85.05 | 80.06 |
A02 | 64.66 | 52.89 | 67.49 | 56.69 | 67.84 | 57.16 | 64.31 | 52.46 | 70.32 | 60.46 | 72.79 | 63.74 |
A03 | 84.98 | 79.96 | 86.45 | 81.92 | 94.14 | 92.19 | 91.57 | 88.76 | 92.31 | 89.74 | 95.24 | 93.65 |
A04 | 71.05 | 61.31 | 69.30 | 60.02 | 74.56 | 66.05 | 70.18 | 60.19 | 72.36 | 63.11 | 75.88 | 67.83 |
A05 | 71.37 | 61.87 | 71.38 | 61.92 | 77.54 | 70.04 | 74.63 | 66.18 | 76.09 | 68.13 | 80.07 | 73.43 |
A06 | 56.28 | 41.63 | 57.21 | 42.95 | 63.26 | 51.06 | 66.97 | 55.97 | 69.30 | 59.08 | 71.16 | 61.56 |
A07 | 83.03 | 77.42 | 84.84 | 79.82 | 86.64 | 82.20 | 85.56 | 80.78 | 87.36 | 83.17 | 91.33 | 88.44 |
A08 | 77.86 | 70.48 | 80.44 | 73.92 | 83.76 | 78.35 | 83.02 | 77.36 | 81.24 | 75.10 | 86.35 | 81.80 |
A09 | 77.65 | 70.20 | 77.27 | 69.66 | 81.82 | 75.72 | 84.84 | 79.78 | 84.47 | 79.30 | 86.20 | 81.59 |
Tab. 2 Comparison of classification performance of L?Net and LH?Net with different data augmentation
受试者 | L‑Net | L‑Net(NA) | L‑Net+(STFT+NA) | LH‑Net | LH‑Net(NA) | LH‑Net+(STFT+NA) | ||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Acc | Kappa | Acc | Kappa | Acc | Kappa | Acc | Kappa | Acc | Kappa | Acc | Kappa | |
均值 | 74.22 | 65.62 | 75.14 | 66.97 | 79.12 | 72.17 | 77.83 | 70.44 | 79.28 | 72.39 | 82.67 | 76.90 |
A01 | 81.14 | 74.86 | 81.85 | 75.81 | 82.56 | 76.73 | 79.36 | 72.48 | 80.07 | 73.43 | 85.05 | 80.06 |
A02 | 64.66 | 52.89 | 67.49 | 56.69 | 67.84 | 57.16 | 64.31 | 52.46 | 70.32 | 60.46 | 72.79 | 63.74 |
A03 | 84.98 | 79.96 | 86.45 | 81.92 | 94.14 | 92.19 | 91.57 | 88.76 | 92.31 | 89.74 | 95.24 | 93.65 |
A04 | 71.05 | 61.31 | 69.30 | 60.02 | 74.56 | 66.05 | 70.18 | 60.19 | 72.36 | 63.11 | 75.88 | 67.83 |
A05 | 71.37 | 61.87 | 71.38 | 61.92 | 77.54 | 70.04 | 74.63 | 66.18 | 76.09 | 68.13 | 80.07 | 73.43 |
A06 | 56.28 | 41.63 | 57.21 | 42.95 | 63.26 | 51.06 | 66.97 | 55.97 | 69.30 | 59.08 | 71.16 | 61.56 |
A07 | 83.03 | 77.42 | 84.84 | 79.82 | 86.64 | 82.20 | 85.56 | 80.78 | 87.36 | 83.17 | 91.33 | 88.44 |
A08 | 77.86 | 70.48 | 80.44 | 73.92 | 83.76 | 78.35 | 83.02 | 77.36 | 81.24 | 75.10 | 86.35 | 81.80 |
A09 | 77.65 | 70.20 | 77.27 | 69.66 | 81.82 | 75.72 | 84.84 | 79.78 | 84.47 | 79.30 | 86.20 | 81.59 |
模型 | 总时间 | 均值 |
---|---|---|
L‑Net | 783 | 87 |
LH‑Net | 944 | 105 |
L‑Net+DA | 1 498 | 166 |
LH‑Net+DA | 1 942 | 216 |
Tab. 3 Comparison of training time of different models
模型 | 总时间 | 均值 |
---|---|---|
L‑Net | 783 | 87 |
LH‑Net | 944 | 105 |
L‑Net+DA | 1 498 | 166 |
LH‑Net+DA | 1 942 | 216 |
1 | LONG J, LI Y, WANG H, et al. A hybrid brain computer interface to control the direction and speed of a simulated or real wheelchair[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2012, 20(5): 720-729. 10.1109/tnsre.2012.2197221 |
2 | ANG K K, CHIN Z Y, WANG C, et al. Filter bank common spatial pattern algorithm on BCI competition IV datasets 2a and 2b[J]. Frontiers in Neuroscience, 2012, 6: Article 39. 10.3389/fnins.2012.00039 |
3 | TONIN L, CARLSON T, LEEB R, et al. Brain‑controlled telepresence robot by motor‑disabled people[C]// Proceedings of the 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Piscataway: IEEE, 2011: 4227-4230. 10.1109/iembs.2011.6091049 |
4 | ACHARYA U R, SREEr S V, SWAPNA G, et al. Automated EEG analysis of epilepsy: a review[J]. Knowledge‑Based Systems, 2013, 45: 147-165. 10.1016/j.knosys.2013.02.014 |
5 | HOSSAIN M S, AMIN S U, ALSULAIMAN M, et al. Applying deep learning for epilepsy seizure detection and brain mapping visualization[J]. ACM Transactions on Multimedia Computing, Communications, and Applications, 2019, 15(1): 1-17. 10.1145/3241056 |
6 | SAFI M S, SAFI S M M. Early detection of Alzheimer’s disease from EEG signals using Hjorth parameters[J]. Biomedical Signal Processing and Control, 2021, 65: 102338. 10.1016/j.bspc.2020.102338 |
7 | ZHANG R, ZONG Q, DOU L, et al. Hybrid deep neural network using transfer learning for EEG motor imagery decoding[J]. Biomedical Signal Processing and Control, 2021, 63: 102144. 10.1016/j.bspc.2020.102144 |
8 | CHIARELLI A M, CROCE P, MERLA A, et al. Deep learning for hybrid EEG‑fNIRS brain‑computer interface: application to motor imagery classification[J]. Journal of Neural Engineering, 2018, 15(3): 036028. 10.1088/1741-2552/aaaf82 |
9 | LAWHERN V J, SOLON A J, WAYTOWICH N R, et al. EEGNet: a compact convolutional neural network for EEG‑based brain–computer interfaces[J]. Journal of Neural Engineering, 2018, 15(5): 056013. 10.1088/1741-2552/aace8c |
10 | SAKHAVI S, GUAN C, YAN S. Learning temporal information for brain‑computer interface using convolutional neural networks[J]. IEEE Transactions on Neural Networks and Learning Systems, 2018, 29(11): 5619-5629. 10.1109/tnnls.2018.2789927 |
11 | AMIN S U, ALSULAIMAN M, MUHAMMAD G, et al. Deep learning for EEG motor imagery classification based on multi‑layer CNNs feature fusion[J]. Future Generation Computer Systems, 2019, 101: 542-554. 10.1016/j.future.2019.06.027 |
12 | 杜秀丽,马振倩,邱少明,等.基于卷积注意力机制的运动想象脑电信号识别[J].计算机工程与应用,2021,57(18):181-185. 10.3778/j.issn.1002-8331.2006-0120 |
DU X L, MA Z Q, QIU S M, et al. Motor imagery EEG signal recognition based on the convolutional attention mechanism[J]. Journal of Computer Engineering and Applications, 2021, 57(18): 181-185. 10.3778/j.issn.1002-8331.2006-0120 | |
13 | SCHIRRMEISTER R T, SPRINGENBERG J T, FIEDERER L D J, et al. Deep learning with convolutional neural networks for EEG decoding and visualization[J]. Human Brain Mapping, 2017, 38(11): 5391-5420. 10.1002/hbm.23730 |
14 | WANG F, ZHONG S‑H, PENG J, et al. Data augmentation for EEG‑based emotion recognition with deep convolutional neural networks[C]// Proceedings of the 2018 International Conference on Multimedia Modeling, LNCS 10705. Cham: Springer, 2018: 82-93. |
15 | SCHWABEDAL J T C, SNYDER J C, CAKMAK A, et al. Addressing class imbalance in classification problems of noisy signals by using fourier transform surrogates[EB/OL]. (2019-01-28) [2021-06-19]. . 10.48550/arXiv.1806.08675 |
16 | ZHANG K, XU G, HAN Z, et al. Data augmentation for motor imagery signal classification based on a hybrid neural network[J]. Sensors, 2020, 20(16): 4485. 10.3390/s20164485 |
17 | HINTON G E, SRIVASTAVA N, KRIZHEVSKY A, et al. Improving neural networks by preventing co‑adaptation of feature detectors[EB/OL]. [2021-09-01]. . |
18 | CHAUDHARY S, TARAN S, BAJAJ V, et al. Convolutional neural network based approach towards motor imagery tasks EEG signals classification[J]. IEEE Sensors Journal, 2019, 19(12): 4494-4500. 10.1109/jsen.2019.2899645 |
19 | BAI S, KOLTER J Z, KOLTUN V. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling[EB/OL]. (2018-04-19) [2021-06-19]. . |
20 | 袁华,陈泽濠.基于时间卷积神经网络的短时交通流预测算法[J].华南理工大学学报(自然科学版),2020,48(11):107-113, 122. 10.12141/j.issn.1000-565X.190276 |
YUAN H, CHEN Z H. Short‑term traffic flow prediction algorithm based on time convolutional neural network[J]. Journal of South China University of Technology (Natural Science Edition), 2020,48(11): 107-113, 122. 10.12141/j.issn.1000-565X.190276 | |
21 | 周飞燕,金林鹏,董军.卷积神经网络研究综述[J].计算机学报,2017,40(6):1229-1251. 10.11897/SP.J.1016.2017.01229 |
ZHOU F Y, JIN L P, DONG J. A survey of convolutional neural networks[J]. Chinese Journal of Computers,2017,40(6): 1229-1251. 10.11897/SP.J.1016.2017.01229 | |
22 | TANGERMANN M, K‑R MÜLLER, AERTSEN A, et al. Review of the BCI Competition Ⅳ[J]. Frontiers in Neuroscience, 2012, 6: Article 00055. 10.3389/fnins.2012.00055 |
23 | SAMIEE K, KOVACS P, GABBOUJ M. Epileptic seizure classification of EEG time‑series using rational discrete short‑time Fourier transform[J]. IEEE Transactions on Biomedical Engineering, 2014, 62(2): 541-552. 10.1109/tbme.2014.2360101 |
24 | ZHAO X, ZHANG H, ZHU G, et al. A multi‑branch 3D convolutional neural network for EEG‑based motor imagery classification[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2019, 27(10): 2164-2177. 10.1109/tnsre.2019.2938295 |
25 | WU H, NIU Y, LI F, et al. A parallel multiscale filter bank convolutional neural networks for motor imagery EEG classification[J]. Frontiers in Neuroscience, 2019, 13: Article 01275. 10.3389/fnins.2019.01275 |
[1] | Yunchuan HUANG, Yongquan JIANG, Juntao HUANG, Yan YANG. Molecular toxicity prediction based on meta graph isomorphism network [J]. Journal of Computer Applications, 2024, 44(9): 2964-2969. |
[2] | Yexin PAN, Zhe YANG. Optimization model for small object detection based on multi-level feature bidirectional fusion [J]. Journal of Computer Applications, 2024, 44(9): 2871-2877. |
[3] | Jing QIN, Zhiguang QIN, Fali LI, Yueheng PENG. Diagnosis of major depressive disorder based on probabilistic sparse self-attention neural network [J]. Journal of Computer Applications, 2024, 44(9): 2970-2974. |
[4] | Xiyuan WANG, Zhancheng ZHANG, Shaokang XU, Baocheng ZHANG, Xiaoqing LUO, Fuyuan HU. Unsupervised cross-domain transfer network for 3D/2D registration in surgical navigation [J]. Journal of Computer Applications, 2024, 44(9): 2911-2918. |
[5] | Shunyong LI, Shiyi LI, Rui XU, Xingwang ZHAO. Incomplete multi-view clustering algorithm based on self-attention fusion [J]. Journal of Computer Applications, 2024, 44(9): 2696-2703. |
[6] | Yuhan LIU, Genlin JI, Hongping ZHANG. Video pedestrian anomaly detection method based on skeleton graph and mixed attention [J]. Journal of Computer Applications, 2024, 44(8): 2551-2557. |
[7] | Yanjie GU, Yingjun ZHANG, Xiaoqian LIU, Wei ZHOU, Wei SUN. Traffic flow forecasting via spatial-temporal multi-graph fusion [J]. Journal of Computer Applications, 2024, 44(8): 2618-2625. |
[8] | Qianhong SHI, Yan YANG, Yongquan JIANG, Xiaocao OUYANG, Wubo FAN, Qiang CHEN, Tao JIANG, Yuan LI. Multi-granularity abrupt change fitting network for air quality prediction [J]. Journal of Computer Applications, 2024, 44(8): 2643-2650. |
[9] | Zheng WU, Zhiyou CHENG, Zhentian WANG, Chuanjian WANG, Sheng WANG, Hui XU. Deep learning-based classification of head movement amplitude during patient anaesthesia resuscitation [J]. Journal of Computer Applications, 2024, 44(7): 2258-2263. |
[10] | Huanhuan LI, Tianqiang HUANG, Xuemei DING, Haifeng LUO, Liqing HUANG. Public traffic demand prediction based on multi-scale spatial-temporal graph convolutional network [J]. Journal of Computer Applications, 2024, 44(7): 2065-2072. |
[11] | Zhi ZHANG, Xin LI, Naifu YE, Kaixi HU. DKP: defending against model stealing attacks based on dark knowledge protection [J]. Journal of Computer Applications, 2024, 44(7): 2080-2086. |
[12] | Yiqun ZHAO, Zhiyu ZHANG, Xue DONG. Anisotropic travel time computation method based on dense residual connection physical information neural networks [J]. Journal of Computer Applications, 2024, 44(7): 2310-2318. |
[13] | Song XU, Wenbo ZHANG, Yifan WANG. Lightweight video salient object detection network based on spatiotemporal information [J]. Journal of Computer Applications, 2024, 44(7): 2192-2199. |
[14] | Xun SUN, Ruifeng FENG, Yanru CHEN. Monocular 3D object detection method integrating depth and instance segmentation [J]. Journal of Computer Applications, 2024, 44(7): 2208-2215. |
[15] | Yajuan ZHAO, Fanjun MENG, Xingjian XU. Review of online education learner knowledge tracing [J]. Journal of Computer Applications, 2024, 44(6): 1683-1698. |
Viewed | ||||||
Full text |
|
|||||
Abstract |
|
|||||